From: Rodolphe Ortalo <Rodolphe.Ortalo@cert.fr>
  To  : ggi-develop@eskimo.com
  Date: Tue, 03 Aug 1999 11:10:22 +0200

Re: Matrox GGI accellerator

James Simmons wrote:

> They know that accel handling, not accel registers belongs in the kerenel.

Yes that's the usual distinction between the 'authorization to use'
and 'real use' of a resources. Of course, initial authorization as well
as regs. access setup should be done by priviledged software (and,
as these resources are hardware resources, it really seems to me
that the target software is the kernel).
The 'normal usage' accesses may be done differently - but well,
it often depends on the hardware design it seems.

> That is the distiction that has to get across. Their has been some
> discussion on this. In fact I had a talk with one of the kernel developers
> today about this. The BIGGEST thing is the need for proper virtualization
> of the accel engine. Security naturally comes out of this. I have been

I understand that such virtualization is very difficult to obtain. In fact,
from all the issues that have been discussed in the past on the
GGI project wrt. that, it seems that a single 'perfect' approach
may not exist. So, we always proposed different techniques.
(And we tried to avoid the true virtualization path because graphics
card engineers are so innovative with the hardware design that.
there are new things all the time it seems. So the idea was
more to propose a flexible software architecture - and I think
the KGI+libGGI combination is usable in most situations.)

> working on the design for /dev/gfx to deal with such issues. I'm modeling
> /dev/gfx after the way it works on SGI workstations. RIght now if I where

SGI hardware is probably the strongest hw design. So you may have the possibility
to cleanly separate access control from 'functional' accesses - that is to
say that you can feasibly define a /dev/gfx interface. But with other
types of hardware it may be difficult.

> to start 3 OpenGL programs in X windows and if all of them bang on the
> registers at the same time it would surely hose the machine. A per process

It's even worse when you consider a malicious program...

> stations. Plus with the window idea you can mmap the zbuffer the size
> of the window. Even if the card only allows a z buffer the size of the
> mode /dev/gfx will only mmap the window area of the zbuffer. This way

To do that you really need very fine grain control. Is it really possible to
mmap any portion of a SGI card framebuffer (at 1 pixel resolution) ? Don't
you have things like 'page granularity' etc. that would hinder the idea?
Note that this idea adresses confidentiality of the fb in fact, no?
Do you really think that window granularity is necessary ? Personnally,
I'd say that a secure application could simply open a _new_ graphic
console. This would prevent a secure and an unsecure application
to display within two windows on the _same_ graphic console. But,
would it be a (security) feature or a bug ? (Your mileage may vary
of course, but the answer is not so evident - to me at least. :-)

> struct gfx_registers {
>         line_x 0x34;
>         line_x_mask 0xff12;
>         line_y 0x35
>         line_y_mask 0xff13;
>         ...
>         ...
> }

Again this requires very fine grain control over hardware access. Is
it really feasible to provide such fine access control (at the bit level
sometimes) without requiring implicitly a kernel mediation on each
access to a register ? If it's possible, this approach is IMHO
pretty good.

However, I suspect that, to provide such mechanisms, in fact, we
need to make assumptions on the way the hardware is designed.
IF the 'safe' accelerations registers are well isolated, say
between 0x800 and 0x8ff (offsets in the MMIO aperture) then
we can allow direct access from userspace in that area. (At least
from the security point of view: safe means that a malicious
program will not burn your monitor or lock the PCI bus.)
BUT this is apparently not the case with 'common' hw.

A single example that I fell upon with the card driver I wrote:
 among the usual 3d acceleration commands of the 546x
(you know: LINE_3D, TRIANGLE, BOX, etc.) there is
also a display list command (REG_ACCESS) that allows
to WRITE in another register. Of course, you can guess that
the 3D acceleration engine does not do access control...
The consequence is that _any_ program that can issue
directly 3D commands to the 3D engine can also, e.g.,
change the screen resolution, etc....

[IMO, this 'feature' was provided so that display lists
could be used to setup a graphic mode - not so stupid
in fact - it could even be convenient...]

With that kind of hardware, virtualization of regs. access
seems infeasible. At least, you know it was not in the
hardware engineers design objectives ! ;-)
(Hence the need for other techniques.)

However, of course, if better hardware allows a clean
policy to be defined and implemented, it may be the
right way to do - but for that hardware, not for all
hardware.

> Hum. Can you expalin in detail the ping pong method. I sort of have a
> idea on how it works but not quite. Thanks.

First:

It seems people always think to virtualization at the registers
level. But in fact, this is not mandatory. E.g. to access the
useful acceleration features of a card like a ViRGE of
Cirrus Logic, you need to virtualize things like
'DRAW_BOX', 'COPY_BOX' (2D) or 'LINE', 'TRIANGLE'
(3D), or TEXTURE_SETUP, etc.

So, first, I'd say that the accesses that could be convenient
may not always be at the registers level. So, a single
command unit may be longer than a byte, it may be a
whole struct.
(I don't say that exporting acceleration registers to
userspace if bad: I say that there may exist different
access granularities that may be satisfying enough not
to bother doing something differently.)

So, in fact, you need a good way to communicate such commands
from the userspace application to the graphic card.

If the command is simply:
   {register_offset, value}
then.... well, you will have a lot of them, and in that situation,
it may be slow, and difficult to control. So, it's typically NOT the
ideal case. (Except for criticism of the whole kernel
mediation interest. ;-)

But if commands look like:
  { DRAW_BOX, X1, X2, Y1, Y2, COLOR }
or {TRIANGLE, ... etc... [1] }
you will have less commands.than in the above case. Especially
when they are 2D commands. With 3D you may have a lot of
commands (1000 triangles for one picture is not unusual) - but
typically they come in a long stream.

Note that, even in this case, you _may_ need to do some
access control (cf: the example I gave you for the 546x).
In case someone tries to use 'strange' accelerators
commands. [2]


Second:

We end up with something that resembles to ioctl, but
the problem is that we face long streams of commands,
and _very_ _very_ impatient users.

So, if, each time one command is issued by the application
software there is a switch to kernel context, an access
control check, and the emission of an accelerated
command to the graphic card before going back to
the application software for the next command....
Well, we end up with only 4x or 5x acceleration in
2D. (No figures available yet for 3D. And that's a
pity.)
[This is the 'ioctl' way.]

However, we can take advantage of the fact that
these commands come in 'stream' to process
them in whole batches.
This is the idea of the pingpong buffer. It's pretty
similar to buffering in fact.
2 mmap pages are setup. But only one of them
is available in userspace. Both are swapped as soon
as the userspace page is filled:
The empty page is put up in userspace and the application
program can continue to put its commands in it.
The other page is handled by the card driver which
reads the commands, controls them, and issue them
to the card.

You can put whatever command you want in that
page (reg+value or complex commands). But the
idea is that the latency of issuing ONE command
will be greatly reduced, and this is very desirable,
especially for 3D.
Now, of course, you will always find someone who
needs something even faster and who wants full
control over this or that register. Well... I'd give
it then (or cancel the account).


When you think to it, the whole system is pretty
adapted to graphic cards programming. In fact,
these 'commands' can be made very similar
to display lists - so the kernel driver may not
really need to interpret or rewrite them, it
simply needs to control them. The latency was
the big problem, and with that scheme, the
limiting factor may be the acceleration engines
hardware fifos (which typically contain a few
commands) or the speed of the accesses from
the card to the display lists. (At least: maybe something
the GGI project will not be blamed for. Well, they
will find something else... :-( )

However, this scheme does _not_ prevent someone
to mmap a full portion of the register to userspace,
and use them directly from there to obtain maximal
throughput - POTENTIALLY bypassing the kernel
driver for any control of course then. (I say potentially
because this may depend on the hardware features, on
the SGI you may setup such interaction without
compromising safety - but frankly, even with SGI
hardware I would be surprised that security will
really be guaranteed [think malicious! ;-)]. On the Cirrus
Logic anyway, forget about both...)

Of course, the main 'drawback' of the whole system
is that it relies on a 'different' software design. Furthermore
it is a rather subtle design, because the flexibility is
the key. Well, now however, you may understand
that the existence of nearly any mechanism inside
KGI or libGGI was motivated by some practical
and pragmatic constatation. I don't say that this
is perfect, but there are some good ideas I think.


Rodolphe (Ortalo)


[1] I don't remember the parameters in fact... You need
the top, left, right and 2 slopes no - and the color or
texture of course ? :-(
[2] Note that both the commands AND the accelerator
are strange. ;-)