GDCE 2004 - Microsoft XNA day- archive

Introduction

The first day was a Microsoft tutorial day dedicated to DirectX and the new XNA initiative.

XNA aims to do make it easier to develop games for different platforms. The current XNA framework supports Windows, XBox and Mobile technologies only but they would like third part developers to create frameworks for the Playstation 2 and Gamecube. To me this is the sticking point - will Sony and Nintendo really want this? The first thing Microsoft are doing is to bring some of the current XBox tools to the Windows platform.

DirectX is to undergo some major changes over the next year or so. Direct3D is now heavily affected by the forthcoming Longhorn release. Longhorn uses Direct3D for hardware acceleration and fancy desktop effects. This means Direct3D will become WGF (Windows Graphic Foundation), GDI will become Avalon.

XACT (XBox Audio Creation Tool) - Dugan Porter

XACT allows composers & sound designers to create dynamic effects and apply real time processing. Most impressive is the in-game auditioning. The programmer makes a variable available that the sound team can use as input into an effect. An example shown was in a racing game - the programmer exposes a variable representing the revs of the car. The sound designer using 3 different engine sounds defines how these sounds will fade in and out dependant on the input value.

XACT 2.0 is in development and will be beta by the end of this year.

PIX (Performance Investigator for DirectX) - Mike Burrows

PIX has been available for XBox development for some time but has now been released for the PC. It is similar to a profiler giving a view of how the game is using the Direct3D API calls and where the time is spent.

PIX uses instrumentation by code interception meaning you do not need to recompile your app and PIX can be run on release code.
If you wish to label parts of your code you can use the D3DPERF set of functions to insert labels. These do not do anything but are picked up by PIX when profiling.
You can also create your own plugin or use ones from ATI and NVIDIA (I saw a demo of the ATI one - very impressive).
The XBox version of PIX is still much better as it can hook into the hardware. So the Windows version is only a subset of what is available on the XBox - this should change with Longhorn. One very impressive function was where you can right click on any pixel on the screen and PIX will show all the functions that led to the creation of that pixel, e.g. shaders etc.
As well as being able to examine Direct3D counters there are many more available from the operating system. There are lots of these like cache miss, memory usage etc.

One side effect of PIX that was quite interesting was that you could see all the Direct3D calls made by the D3DX library.

Other issues

Only 25% of the time spent in a game is in the API.
NTFS is very fast for large files but not so fast for small files - need to bare this in mind when writing loader code.
Programmers should use Unicode nowadays instead of ANSI.
By default you should optimise for size not speed! This one surprised me - I think I will carry out some optimisation tests to see if this true.
Microsoft are now expecting to update the DirectX SDK every other month. Note that this does not mean they will be upgrading the runtime library so quickly - just the SDK.

Effects Best Practice - Craig Peeper

A talk describing the technology behind the D3DX effects system.

BeginPass is a good place to optimise
CommitChanges and BeginPass are the only two commands that actually apply state changes
Should sort by effect for best performance
Try to minimise the number of calls to BeginPass and maximise calls to CommitChanges
The constant table can be used without a performance hit
A pre-shader is code that is constant at run time - extracted from a vertex or pixel shader. Pre-shaders are enabled by default. What happens is that code with a constant result can be calculated in advance and removed from the shader., This all happens automatically.
Effect state manager - can write user callbacks that will be called by the effect

Common Preview Pipeline - Kee Gee

The bottleneck for games development has now moved from the engine technology to the creation of game content. Art teams have often grown exponentially to the programming teams. The common preview pipeline is a set of plugins aimed at content creation. Shaders are now common in games and the artist needs to be able to see the affect as they edit their creations.

Integrated tools in Maya - export to .x format
A new UI for editing effects variables in shaders - gives controls to the artist for editing shader values allowing the artist to tweak variables e.g. specular highlight (see also below)
There will be a new file format built on the .x file format called the X2 format
.x file now has a custom effects template.

Creating Content Using Standard Annotations & Semantics - Kev Gee

The latest DirectX SDK (Summer 2004) introduced a way of providing grammar for dealing with HLSL shaders. This then links in with the preview pipeline as variables can be exposed to the artist.

An annotation is meta data for a parameter that can be read easily and used by art tools. A semantic is how the data is bound to the shader. In addition there will be a new scripting system - still being defined. This will allow the programming of logic into the effect file. Standards are used in FX composer. Rendermonkey is more of a shader editor than composer but can export to FX. You can add your own annotations for your own use - not just use the built in ones.

Microsoft High Level Shader Language (HLSL) - Craig Peeper

This talked introduced the latest developments made to HLSL to support the latest shader models.

Pixel Shader 2.0b is improved
PS 3.0 adds what was possible in 2.0 only via the caps.
It is also 32 bit.
Has looping but not using constants - branching is not amazing
New registers:

vFace is a new register allowing for double sided lighting
vPos gives actual pixel rasterization position

Can only use PS 3.0 with VS 3.0 you cannot mix the two
New commands

Addition of centroid - a way of doing interpolation with multiple sampling. Can use change in value with dsx and dsy gradients
TexDD allows the gradients of a texture to be used. VS 3.0 gives a lot more registers and texture look-ups. No gradients at this stage. Can do texture look up with LOD specified but this can be slow versus simple texture look up.

Practical PRT - Peter-Pike Sloan

PRT stands for Precomputed Radiance Transfer and is a technique that enables complex global illumination effects like soft shadows, inter reflections & subsurface scattering. I found this to be pretty impressive and relatively simple to implement.

Traditional lighting uses the dot product method which gives only a very rough approximation. PRT for smooth low frequency lights. Transport complexity is how light interacts with scene e.g shadows, inter reflections, caustics translucency (subsurface scattering). Source light radiance coming from a spherical environment map. 3 techniques.

1. Irradiance environment map. Irradiance means integral of all incoming light at a point. A diffuse cube map is precomputed which is in the spacial domain. Can also do in frequency domain (use D3DX for this complex stuff). Spherical Harmonics is basically a compression system for lighting. Sample lighting environments in a scene and interpolate between them. This method is light weight compared to PRT. The two can also be combined if required.

2. PRT. Came originally from a Sigraph paper. Enables complex light transport combined with dynamic lighting environments. Provides soft self shadows, interactions and colour bleeding etc. Light environment is represented with spherical functions which makes things easier. You should always compress PRT transfer vectors. You can do adaptive sampling where vertices are added just to resolve shadows. The limitations of PRT are that the shadows are soft and the light is considered to be distant.

3. Local Deformable PRT. Animating PRT is difficult. LDPRT approximates a transfer vector. Can replace PRT and normal maps but is less accurate. Uses bent normals giving a better blob.

These notes do not describe these processes very well, I suggest you take a look at the slides and some PRT demo's to understand these processes more fully.

DirectX Futures - David Blythe

Currently the DirectX SDK contains the following APIs

Direct3D - graphics
DirectSound - sound
DirectMusic - music
DirectPlay - networking
DirectInput - input

With the approach of Longhorn and the introduction of the XNA tools DirectX is going to be split up:

Direct3D will become the Windows Graphics Foundation (WGF) and will provide the base for the Longhorn desktop.
DirectSound and DirectMusic will be replaced by XACT and LEAP (see below)
DirectPlay will be replaced by XBox live technology
DirectShow will be replaced by a new media SDK
DirectInput, I believe, will stay as it is.

In most cases the current state of the APIs will remain but will no longer be developed. New facilities from XNA will be added e.g. DirectPlay will remain as it is but the XBox live technologies will be added.

WGF will see the end of most of the fixed function pipeline e.g. vertex lighting, fog, alpha testing, FVFs, triangle fans, legacy texture formats and point sprites are all being removed. There will be a common shader core that will bring vertex and pixel shaders closer together. Microsoft would also like to get rid of the DEV caps. These caps were added to Direct3D when requested by the hardware manufacturers (IHVs). It meant they could provide a new facility without a complete change to the Direct3D specification. This means that IHVs will have to match the specification of WGF which may not please them as they often want to add their own extra capabilities.

The new driver modal in Longhorn will move a lot of the driver processing from the kernel into the user mode. This should mean the elimination of Blue Screens Of Death (BSOD) due to driver failures.

Additional features will include:

Improved state management,
High-definition rendering formats e.g for high dynamic range lighting,
Normal map compression
The addition of texture compare filters
New element to the graphics pipeline for processing more complex geometry, which may in turn lead to the use of the GPU for non-traditional tasks, such as collision detection or physics.

The bad side of things

Development times too long
Shader model 2 adds a lot of new capabilities
When a Windows XP program crashes you get the opportunity to send a bug report to Microsoft. From these statistics previously they were seeing that 25% of all crashes were video card related. They have now got this down to about 17% but hope in Longhorn, with a new driver modal, to decrease these further.

Longhorn Changes and later

The GPU will no longer be allowed to be grabbed by one program but will be available to multiple applications at the same time. The graphics hardware will be virtualised e.g. virtual GPU memory
All applications in Longhorn will write to a back buffer then these back buffers will be composited together.
Worth looking at the WGF rendering pipeline - I believe it is available on MSDN but I hope to add a link here soon.
The next version will have a common shader core so PS and VS will look code wise the same using just the one instruction set.
D3D is just a plumbing tool all the real API is in HLSL.
Other future ideas are subroutines shared between shaders. Evolution of effects framework into authoring packages, runtime applications etc.
Geometry shaders - e.g. Silhouette edges, extrusion, fins fur etc, could also replace point sprites
Allow manipulation at last of x,y co-ordinates in shaders

Longhorn WGF to go beta in 2005.

New DirectX SDK update at end of this year.

Conference Chatter

The general feeling about the changes to DirectX were quite positive. I myself was a bit concerned that Longhorn using Direct3D for the desktop rendering would mean the Direct3D would be more driven by the OS than by games developers. The new XNA tools like PIX and XACT were well received - a lot of people already had very positive reports of using PIX on the XBox.

Microsoft seemed to be apologetic (and perhaps embarrassed?) about the mess up with ATI and the latest DirectX. Only the NVIDIA 6800 currently supports version 3.0 of the shaders. Apparently communication between ATI and Microsoft was confused and with the need to release the latest runtime library in SP2 there was no time to resolve the problems. However Microsoft say they are working on a solution to give better support to the ATI cards.