From: Steffen Seeger <s.seeger@physik.tu-chemnitz.de>
  To  : GGI GGI <ggi-develop@eskimo.com>
  Date: Fri, 23 Apr 1999 11:22:16 +0200 (MEST)

KGI and UDI

Hello everybody,

I tried to send this message yesterday, but I seem to have some problems
sending e-mails from my upgraded system.

Anyway, I have some news to share that will influence our work in the next
time considerably, I think.

I have read through the UDI spec lately, which is the reason for no coding on
kgi last week. However, I noticed they did not yet include a display driver
specification and contacted Mark Evenson what their plans are and what we
did so far.

The responses:
Mark Evenson, mevenson@cup.hp.com wrote:
> I took a quick glance at the GGI web page, and it looks like the GGI
> work dovetails very well with what we're doing in UDI.  We would like
> to include a display hardware driver specification in UDI, and would
> welcome working with you and your group to make that happen.

Kurt Gollhardt, kdg@sco.com wrote:
> This is very good news! I had not been aware of the GGI project, but
> there has been a lot of interest in the idea of a standardized graphics
> driver interface that's come up in conjunction with UDI. I look forward
> to working with the GGI project to bring these technologies together.

I wrote up an article giving a rough overview about KGI in general
(attached to this mail) and have been invited to participate in the 
UDI teleconference.

We agreed to proceed as follows:

0)	The UDI people will support our efforts making KGI drivers
	UDI-compliant, and we will contribute to the UDI display driver
	specification.

1)	We both need to become familiar with the other project.
	Some of the UDI group like to join the KGI Mailing list to help
	us getting into UDI, while we will help them into KGI.

2)	Both projects work on a prototype implementation of their ideas
	right now, and we should get that to a working state first.
	A UDI sample implementation will be released to the public
	domain when ready, however, I might get access to it earlier
	but will have to sign some paperwork.

3)	Meanwhile we have to figure out what parts of KGI are candidates
	to be replaced by UDI, what parts have to become part of the
	UDI specification and what parts remain parts of the KGI 
	specification. Especially we need to figure out right now if
	the UDI core specification is sufficient or needs to be extended
	to handle graphics hardware the way we would like/need it.

4)	After doing this, we will work together to work out the details
	of the specifications, and getting a sample implementation to work.

So, basically this mail is to announce that Project UDI and The GGI Project
will undertake a joint effort to specify a portable display driver model
and environment. I think I speak for everyone here when saying that I would
like this to be an open effort, like GGI and UDI have always been.
I invite everyone willing to participate to help us getting it right.

Finally I would like to thank anyone who helped to get GGI that far,

			Steffen

PS: Administrative details to follow soon.

----------------- e-mail: seeger@physik.tu-chemnitz.de -----------------

Hello,

I'm a member of a project called 'The GGI (General Graphics Interface) Project'.
Part of this project is the development of a standardized environment for 
display hardware drivers called KGI (Kernel Graphics Interface). 

A lot of the underlying concepts and goals are similar to UDI, which is 
why we are very interested in integrating our architecture and the UDI
framework. As a first step, I have written a white paper on the KGI 
portion of GGI that the UDI community can look at to get an impression 
of the KGI operation and concepts behind.

As a second step I am making myself familiar with UDI. If desired I can
write a second article giving some more technical details about the
display driver part of KGI and my ideas how to integrate it into UDI.
However, I need to get a better overview of UDI first.

Feel free to send me any questions or comments. I am also available for
questions/discussions at todays teleconference.

			Steffen

------------------ e-mail: seeger@physik.tu-chemnitz.de -------------------
--------------- The GGI Project: http://www.ggi-project.org ---------------
----- for KGI snapshots see: http: http://www.tu-chemnitz.de/~sse/ggi -----

An overview of KGI - a Kernel Graphics Interface.
=================================================
written by Steffen Seeger <seeger@physik.tu-chemnitz.de>


Abstract
--------
This draft is about the Kernel Graphics Interface developed by the
GGI Project. It is intended to give a rought overview of the concepts
behind, being kind of a white paper. It'll take about 20min to read, 
if you like to know more about it or comment on it, feel free to email
the author. You can download a snapshot of the current sample 
implementation from http://www.tu-chemnitz.de/~sse/ggi/kgi-0.9-990405.tar.gz
A sample driver implementation for the 3Dlabs Permedia chipset family
is being implemented right now, a non-working snapshot is available from
the author upon request. Any file references given refer to files 
included in those snapshots.

0. Introduction
---------------
The development of a Kernel Graphics Interface (KGI) is part of the 
GGI Projects effort to create a portable, fast and secure 
General Graphics Interface (GGI). While GGI itself is more a suite
of application programming interfaces, most applications will
not directly interface to KGI. Instead, KGI is intended to provide
the neccessary kernel support (protection, virtualization and
abstraction) in order to allow applications to benefit 
from graphics hardware with as little overhead as possible.
Additionally, a certain amount of kernel level support for display 
hardware can ease the effort to meet the security and stability 
requirements of modern multi-user multi-tasking operating systems.

1. Display Hardware Characteristics and KGI design requirements
---------------------------------------------------------------
As mentioned, KGI deals primarily with display hardware. This section
should give some fundamental properties of graphics hardware and their
implication on KGI design. Some of these properties make it special
compared to other computer subsystems, while others are just implied
by the fact that it is hardware. Most of the KGI design goals and
constraints can be summarized as:

     +--------------------------------------------------------------+
     | Display hardware is essential for human-machine interaction. |
     +--------------------------------------------------------------+

This very obvious property makes display hardware different from
all other computer componentes and implies the following design 
requirements for KGI:

0) "Display" ...
	Display hardware must become available very early during
	the boot process in order to display diagnostic messages.
	KGI must therefore provide means to output diagnostic 
	messages from the kernel and do this as early as possible.

1) ... "hardware" ...
	The primary intention of advanced display hardware (other
	than simple frame buffers) is to offload computing or
	memory bandwith intensive operations to specialized
	(co-)processing hardware, thereby freeing the main CPU
	for other tasks. In some aspects, display hardware therefore
	acts as a co-processor that needs to be virtualized - 
	just like the main (co-)processing unit(s) (e.g. FPU, ...).
	On the other hand, display hardware is also a system
	resource that needs to be protected from accidental
	or intentional misuse. Finally, KGI drivers -- or firmware --
	are (as far as common features are concerned) considered 
	part of the display hardware and must be designed to be
	independent of the OS and host architecture.

2) ... is "essential" for ...
	Display hardware must be reliable. This implies that the
	user or operator should be able to get access to a 
	'rescue shell' even if graphical user interfaces are not
	accessible. (E.g. the graphical user interface sever crashed.)
	Therefore KGI must not rely on applications to free resources
	properly.

3) ... "human-machine" ...
	Display hardware is built for humans working with computers.
	This requires KGI neither to prescribe nor to prevent
	policies if/how to share display hardware, what to
	do in case of non-fatal errors etc.

4) ... "interaction".
	Display hardware must be responsive. This implies that
	even for highly loaded machines delays between user input 
	and visible feedback due to this change must be short. With
	3D graphics becoming more and more standard, this requires
	advanced hardware programming techniques, such as graphics
	processor virtualization, priority scheduling, "processes 
	models" for graphics processors etc. KGI must therefore
	provide means to handle several processes sharing resources
	where this makes sense.

These requirements are accompanied by the fact that display hardware is 
not standardized and probably the fastest evolving computer subsystem.
KGI should therefore be open for future developments and not be specific
to a particular graphics application programming interface. And last but
not least, KGI is intended for UNIX-like operating systems, which feature
multi-user multi-tasking environments. Thus KGI should allow for simple
setup of independent multihead machines, featuring several workplaces
per machine.

2. The KGI Architecture
-----------------------
The KGI architecture can be structured into the following 'subsystems':

	- a human-machine interaction model
	- an abstract display hardware model
	- physical I/O and environment operating system support
	- display hardware drivers and management
	- input hardware drivers and management
	- application interface (graphics process management)

Additionally the current KGI sample implementation for Linux has also
a console subsystem and compatibility module, adopted to the changes
neccessary to implement the KGI architecture. 

The subsequent text will focus on the interaction and hardware models,
OS environment, display driver and application interface. The input
hardware driver, console and compatibility module will not be
described in much detail.

2.1 The KGI Human-Machine Interaction Model
-------------------------------------------
KGI is developed for multi-user multi-tasking operating systems
that allow several users to run processes simultaneously, providing
means to isolate (protect) processes from each other, but also to 
share limited resources. KGI is consequently designed to allow several
users to work interactively with the same machine. It allows each user
to use an arbitrary number of display and input hardware (often
referred to as independent multi-head). Input and output devices
are grouped to "workplaces" owned by a particular user (the one using it).
In order to access resources of a particular workplace, processes must
belong to the user or have explicitly been granted access to these
resources.

NOTE	Actually all human-machine interaction hardware like sound,
NOTE	workplace local peripherals, etc. needs to be treated this way.
NOTE	However, KGI is currently limited to input and visual output.
NOTE	Actually, Kernel User Interface describes better what KGI does,
NOTE	but originally KGI was intended to handle display hardware only.
NOTE	Also, it should	be pointed out clearly, that this Kernel User
NOTE	Interface is restricted to hardware drivers, protection and
NOTE	virtualization. Especially it must not be misunderstood as
NOTE	running a Graphical User Interface at kernel level, which 
NOTE	is a common misunderstanding about KGI.

Additionally, KGI allows for virtual (textual and graphical) consoles 
that are mapped on physical display and input hardware (thereby 
becoming visible/active).

Analysing the flow of information gives a very natural model of human-machine
interaction. Information going into the machine is "digitized" by input
hardware. This digitizing is dealt with by the Kernel Input Interface 
file:kgi/include/kii/kii.h. Information going out of the machine is 
converted by output (display) hardware. This is dealt with by the 
Kernel Graphics Interface file:kgi/include/kgi/kgi.h. The actual 
information processing is done in processes that interface to hardware
through abstract hardware representations, referred to as devices. 
Devices virtualize limited hardware resources.

All input device drivers (event sources - <struct kii_input>) register 
with a central mangagement code (file:kgi/src/Linux/kgi.c) that 
enumerates input devices and groups these into so-called focuses 
<struct kii_focus>. All input devices associated with a focus send 
events to the same focused object (e.g. a virtual console) represented
by a device (event-sink) <struct kii_device>. The focused object then processes
the event, the result being commands sent to the display hardware
represented by a device <struct kgi_device>. These get handed to the
display hardware driver <struct kgi_display>, which in turn sends commands
to the hardware, usually resulting in some feedback to the user.

For both KGI and KII care has been taken to separate hardware device driver
specific issues (kii_input, kgi_display) from logical device representation
(kii_device, kgi_device).

2.2 The Display Hardware Model
------------------------------
Despite all differences between the various graphics hardware, ranging
from simple text frame buffers (Monochrome Display Adapter, LCD panels for
embedded systems, etc.) up to virtual reality hardware as SGI's 
"Infinite Reality", certain properties of graphics hardware can be
modeled and abstracted. KGI's display hardware model is intended to give
a functional representation of the underlying hardware, hiding implementation
details from applications.

Attributes
----------
Display hardware controls some attributes of the visible display area 
either independently for small sub-areas (e.g. the color of a pixel) or
the whole display area (e.g. brightness). KGI categorizes attributes into

	- attributes that can be controlled independently per pixel, left
	  and probably right frame (e.g. color, alpha value)
	- attributes that can be controlled independently per pixel, but
	  common to all frames (e.g. z-value, stencil masks, ...)

Sometimes the number for bits stored per attribute needs to be given,
which is done using a bitmask indicating which attributes are present
and a zero-terminated array of 8bit unsigned integers. 
(file:kgi/include/kgi/kgi.h gives details about the attributes defined.)

Images
------
Images are a rectangular set of pixels, the attributes of those being
stored as an array of unsigned integers (the frame buffer). Not all of 
the bits stored per pixel may control visible attributes (e.g. z-values).
Certain attributes (usually color or color index value) are read from 
the buffer in a certain order, control the properties of a given visible
image unit (dot) and serialized into a dot stream. There might be some
scaling and interpolation units or look up tables before the final dot 
stream assembly, which is why KGI distinguishes between image units
(pixel) and dot stream units (dots).

Dot Stream Converters and Dot Ports
-----------------------------------
The dot streams generated by several image units (e.g. a GUI frame buffer
and a video overlay buffer) are passed to some conversion device or
processing unit before being converted into a sensable physical property.
These devices are referred to as "dot stream converters" (e.g. a overlay
processor, video DAC, etc.). KGI represents these devices by a dot port,
which is a description of the transfer protocol or dot stream format.
A dot stream converter is simply a number of input dot ports an one
output dot port. Several dot stream converters may be connected, 
building a tree with dot stream converters as nodes and images as
leaves. The root node is a dot port that represents the monitor.

Display Modes
-------------
Modern display hardware can be operated in a great variety of modes, e.g.
color indexed mode, text mode, true color mode, true color with video 
overlay etc. Instead of enumerating and standardizing these modes (like
1024x768 pixel with 8bit Red, 8bit Green, 8bit Blue and 8bit Alpha),
KGI provides means to check if a certain requested mode is possible or not.
Roughly speaking, a mode describes the hardware operation mode (number of
images, attributes per image, dot stream converter properties, etc.).

Display Resources
-----------------
As a result of a successful mode-check, the hardware driver(s) will return
a set of resources available in this mode. While images, dot streams and
dot stream converters describe the transformation process from the internal
buffer representation to a visible image, display resources provide means to
alter image buffers and internal buffers (like texture memory).
Currently KGI specifies memory mapped I/O regions and accelerators.
It is planned to extend these to support the VESA Media Interface (VMI),
card-local I2O buses, proprietary buses (S3 Streams Interface), etc.

Some resources are static, e.g. video input and output buffers, and may
influence the number of possible modes. Other resources are mode-dependent,
e.g. may only be availiable in certain modes or operating system environments.
For each resource access policy restrictions are given, describing the
hardware driver's constraints about sharing this resource (exclusive maps,
write/read re-ordering, allowed access widths, etc.). Some display
resources (e.g. frame buffers) may be exported directly to applications,
allowing for fast access and high performance. Other resources 
(e.g. accelerators) are not directly exported, but require some buffering
and context management mechanism. This is described in more detail in
the Application Interface section below.

2.3 Display Drivers and Display Mangement
-----------------------------------------
KGI display drivers bridge between the display hardware and the application
interface. For the most common hardware design, a modular driver architecture
has been developed to allow for maximum re-usage of coding efforts.
The most common design of display hardware can be sketched like this:

	                     +---------+
 	                     | Timing  |
	        +------------|Reference|--------+
	        |            +---------+        |
                |                 |             |
	+--------------+    +-----------+    +-----+      +-------+
	|Frame Buffer  |/==\|Accelerator|/==\|Video|----->|Monitor|
	|Texture Buffer|\==/|  Chipset  |\==/| DAC |----->|Display|
	+--------------+    +-----------+    +-----+      +-------+
				|||||
	                    Host Interface

Though recent developments allow the integration of some parts into one
piece of silicon, the functional units can still be identified. Each functional
unit (except memory) is represented by a driver module, using a well
defined internal interface. A fully operational display hardware driver
is obtained by linking the proper (binary) subsystem drivers together.
For example, a graphics card based on a 3Dlabs PermediaNT, with a TI TVP 3026
ramdac with integrated clock chip and a multisync monitor may reuse the
ramdac, clock and monitor driver written for a S3 Vision968 based board with
TI TVP 3026 and multisync monitor.

NOTE	A proof-of-concept implementation has been released by the GGI Project
NOTE	in early 1998. Valuable feedback and conclusions have
NOTE	been drawn from this implementation, leading to an overhaul
NOTE	of the internal module interface. A sample driver for the 
NOTE	architecture described in this document is being developed, but
NOTE	not operational yet. However, it is developed far enough to give
NOTE	a impression how to use the internal module interface.

As KGI display drivers may run in kernel mode, the use of floating point
operations is not allowed. All drivers must conform to certain requirements,
such as they must not call other than the KGI environment functions. Also,
KGI drivers must not rely on certain advanced features such as interrupt
handling, DMA support etc. being available. However, if present they
may benefit from this. This allows KGI driver to be used as userspace drivers
or in-kernel drivers without modification.

The intra-display driver interface is defined in 
file:driver/graphics/include/kgi/module.h

2.4 The Physical I/O and Environment OS Support
-----------------------------------------------
A main goal for the display driver modules was to be independent
of the underlying Operating System Environment. This requires the usual
abstraction of system-dependent data types, physical I/O methods,
IRQ/DMA handling and environment. The system dependent data types and
the environment functions (memset, etc.) are considered straightforward 
and will not be discussed in detail here.

I/O regions
-----------
Physical I/O is abstracted by so-called regions, which are a physical
contigous part of an address space. For each address space, the following
abstractions/operations apply:

- for each address space virtual, physical, I/O and bus addresses are
  defined. Virtual addresses are used by the driver access this region. 
  Physical addresses are used to establish mappings of this region into other
  virtual address spaces. I/O addresses are the addresses that the device
  will respond to if applied at the address select lines. Bus addresses
  are used to access the device from other devices on the same bus.
  (E.g. a frame grabber writing directly to the frame buffer.)

- before accessing a region, the I/O address of a device must be specified
  and the region must either be allocated (allowing auto-configuration) or
  claimed. If this is successful, the driver is granted exclusive use of
  this region. Upon driver termination or if not neccessary for operating the
  hardware, the allocated regions must be freed again.

- The driver may only access the region using the virtual address obtained
  during registration and special I/O functions. Currently defined are
  simple input and output (read/write) operations, as well as copy and
  string input/output operations. Some operations may not be defined,
  depending on the capabilities of the I/O bus system.

Subregions may be exported to application or other drivers, e.g. a 
driver may claim a region of 64MB, containing 32MB frame buffer and
32MB control registers and export only 32MB to the application using
a memory mapped I/O resource. For details of the definitions, see
file:config/include/ggi/io.h.

2.5 The Application Interface
-----------------------------
Applications do not interface to graphics hardware directly, unless approved
by the display driver. Instead, an abstract representation of graphics 
hardware is defined and accessed by applications using a special device
file and standard UNIX file operations (open, close, ioctl, mmap, read).

open	are used to maintain virtualization of display hardware resources.
close

ioctl	is used to control overall behaviour of display hardware and to send
	commands to the hardware (e.g. setting of look-up tables, control of
	video overlay scaling, etc.).

read	is used to report asynchronous events (accelerator programming error,
	accelerator finished, request for shared mapping from other processes
	etc.)

mmap	is used to gain direct access to accelerator command buffers, memory
	mapped I/O regions, such as video stream ports, accelerator register
	files, frame buffer regions etc. Applications that want to use direct
	access may be required to conform to certain restrictions, e.g.
	access only with certain widths, etc. This may be indicated by
	setting access permissions for the special file and using set-user-id
	features, or implementing some verification protocol with the session
	manager. KGI tries not to impose policy here, except that the driver's
	constraints have to be met.

3. Summary and State of Development
-----------------------------------
This article was intended to give an overview about the KGI architecture to
handle display hardware. Regarding the driver section, some of the main goals
and concepts are very similar to those used by the UDI Project.

Currently the GGI Project is working on a improved proof-of-concept 
implementation (kgi-0.9) using Linux as a target OS, that will implement 
the enhanced concepts presented in this articles, such as graphics 
processor virtualization as part of a first version of a direct-rendering
architecture. This will include an experimental X server (XKGI) adopted 
to this architecture.

We are very interested in porting our drivers and architecture to other
platforms, and look forward to use the UDI specifications as a basis for a 
redesigned, UDI compliant driver specification.

The GGI Project would like to submit the KGI architecture as a proposal for
a UDI display/user interface hardware metalanguage. Though doubtlessly some
work is neccessary to integrate both architectures, we hope this will be a
base for fruitful discussion and collaboration between both projects.