IRCAD (2002 - 2003)

• Institut de Recherche contre les Cancers de l'Appareil Digestif
• Startup
  • Virtual-Surg team
• Augmented Reality Research Engineer

Dassault Systèmes (2003+)

• 3D Visualization Engineer
  • Scenegraph, Materials
  • Geometry, Tessellation
• Virtual and Augmented Reality (XR) Engineer
• XR Research Engineer
• XR Research Manager

Dassault Systèmes

From Shape to Life

Course audience

• Computer Science students learning Computer Graphics, Physics or Machine Learning
  • and who are afraid to ask "what's a GPU?", "what's a shader?"
• Web designers wishing to add 3D graphics to their sites
• Game developers wishing to conquer the Web
• Anyone looking for a simple introduction to 3D
  • and who has no clue where to start

Feel free to skim through technical sections and use this course as future reference

Course prerequisites

We'll start from scratch but these should help:

• Math
  • 3D vectors and matrices
• Programming
  • JavaScript notions, or any similar language (HTML kept minimal)
• 3D API
  • OpenGL, DirectX, Metal
• 3D Software (Blender, Unity, Unreal Engine, Godot Engine)
• Desktop / Laptop + VSCode

Course contents

• Demo
  • Existing 3D Web Apps
• Theory
  • A brief history of 3D on the Web
  • Concepts: Architecture, Pipeline, APIs
• Practice
  • 3D Web Programming
    • WebGL, THREE.js
    • Other APIs

Planning

• Session 1 (2 hours)

  • Theory (25 min)
  • WebGL exercises (10 min)
  • THREE.js Theory + full exercise (1h10)
  • Explore examples + choose a personal project (15 min)

• Session 2 (4 hours)

  • Project kick-off (must be finished at home )
    • 2 people per project: clear responsibilities (who does what)
    • send git repo link: source + live testing

Project evaluation criteria

• originality
• interactions
• physics / animations / sounds / eye-candy
• healthcare
• code quality , tricks , performance
• fun

Grading system

• 20 points maximum
• choose features from previous slide
• for each implemented feature:
  • not done: 0 pt
  • nice try / buggy: 1 pt
  • basic / good enough: 2 pts
  • great / polished: 3 pts
  • impressive: 4 pts

Minecraft Classic

Aviator

Aviator 2

Heraclos, (Gobelins)

Blob Opera (Google)

Apple iPhone Studio

BMW

OUIGO

SculptGL

3D Sculptor (Dassault Systèmes)

OnShape (PTC)

SketchFab (Epic)

Zygote Body

Z-Anatomy

NASA

The Cursed Library

Prehistory (1983 - 1993)

• Silicon Graphics (SGI) hardware only
  • IRIX OS
• IRIS GL (1983)
  • API close to hardware
• IRIS Inventor (1988)
• OpenGL 1.0 (1993)
  • Open API, Multi-OS

Fixed Pipeline (1993 - 2004)

• 3dfx Glide API (1996)
  • Voodoo: "hardware 3D acceleration" for all
• Microsoft Direct X API (1997)
  • Windows-only
• OpenGL ES (2004)
  • Subset for "Embedded Systems"
  • "most widely deployed 3D graphics API"
• OpenGL 2.0 (2004)
  • GLSL Shaders

• Foundation of nVIDIA (1993)
  

• NV1 in SEGA Saturn (1994)
  

• GeForce 256 (1999)

  • democratizes the GPU: Graphics Processing Unit, Transform & Ligthing

• GeForce 3 (2001): NV2A in Microsoft's Xbox, programmable shading
  

Shaders, Mobile, Web (2004+)

• OpenGL ES 2.0 (2007)
  • Mobile subset with shaders
• Canvas 3D (2007), WebGL ancestor
  • created by Vladimir Vukićević at Mozilla
• WebGL 1.0 (2011)
  • OpenGL ES 2.0 functionality for the Web!
• OpenGL ES 3.0 (2012), 3.1 (2014): not for Apple
• WebGL 2.0 (2017)
  • OpenGL ES 3.0 exposed to the Web

WebGL architecture: software stack

• Code: HTML + CSS + JS
  • JS code inside the web page makes WebGL API calls
• Browser:
  • browser interprets JS code (using JS Engine)
  • turns WebGL calls into OpenGL calls (binding)
• OS + Driver: converts OpenGL calls to
  • DirectX calls on Windows, Metal on Apple (using ANGLE)
  • OpenGL or OpenGL ES calls on other OSes
• CPU + GPU: run the hardware accelerated code!

Binding example: from JS to C++

gl.drawElements(primitiveType, count, indexType, offset);

JSValue JSCanvasRenderingContext3D::glDrawElements(JSC::ExecState* exec, JSC::ArgList const& args)
{
    unsigned mode = args.at(0).toInt32(exec);
    unsigned type = args.at(1).toInt32(exec);
    
    unsigned int count = 0;
    
    // If the third param is not an object, it is a number, which is the count.
    // In this case if there is a 4th param, it is the offset. If there is no
    // 4th param, the offset is 0
    if (!args.at(2).isObject()) {
        count = args.at(2).toInt32(exec);
        unsigned int offset = (args.size() > 3) ? args.at(3).toInt32(exec) : 0;
        impl()->glDrawElements(mode, count, type, (void*) offset);
    } else {

Impact of stack on performance

• Interpreted JS code is ~10x slower than native code
  • unless you use WebAssembly ("only" 2x slower than native)
• On mobile devices, native code is ~10x slower than on desktop
• Performance tips
  • reduce processing in JS code, let the shaders do the hard work
  • once shaders are compiled and rendering data is on the GPU, the code runs at near native speeds
  • GPU memory is limited: use Draco geometry compression, and Basis GPU texture compression

Evolution

Evolution issues

• New hardware, new needs since 1993
  • mobiles, wearables
  • embedded systems, AI, Vision
• API has become more and more complex
  • coding fast and bug-free drivers is hard
  • OpenGL extensions are not universal
  • API subset needed to deprecate old and slow APIs
• New API needed, and it should be
  • low-level, universal, fast and abstract

But do we really need a new API?

• Not really, see AZDO: Approaching Zero Driver Overhead (2016)
  • using the "right" OpenGL subset and the "right" extensions, we can squeeze as much performance as possible from the GPU
    • https://fr.slideshare.net/CassEveritt/approaching-zero-driver-overhead
    • https://fr.slideshare.net/tlorach/opengl-nvidia-commandlistapproaching-zerodriveroverhead
  • main idea
    • free the CPU fewer DrawCalls
    • keep the GPU busy send more data at once

API Fragmentation

• Proprietary API proliferation no portability
  • close to the GPU fast
  • new "clean"
• Direct X 12 (Microsoft) : Windows, Xbox
• Mantle (AMD) transferred to Khronos to become Vulkan
  • new open low-level standard
• Metal (Apple)
  • looks like Vulkan, Metal was created first

WebGL Next == WebGPU ?

• Apple's "WebMetal", API similar to Metal
• API partially reused and renamed WebGPU
  • temporary name, API still being defined
  • both low-level and object-oriented (no global state!)
  • fast
  • subset for web AND native, despite the "web" in the name
    • a bit like "Vulkan ES" / "Metal ES"
• Might replace WebCL (Compute Shaders), abandoned
• Compatible with WebAssembly

WebGL 1.0

• available to 98,45% of users!
  • data from January 2024: https://caniuse.com/webgl
  • even 99.72% according to https://web3dsurvey.com/webgl
• including mobile browsers
  • Chrome for Android
  • iOS Safari

available everywhere!

WebGL 2.0

• available to 96.34% of users!
  • data from january 2024: https://caniuse.com/webgl2
• even 97.92% according to https://web3dsurvey.com/webgl2
• Standard at Apple since iOS 15! (september 2021)

official, you can start coding with it! (check availability)

• retrocompatibility: WebGL 1.0 works in a WebGL 2.0 context
• WebGL 1.0 polyfill to support a subset of WebGL 2.0 ( shaders)
• you should learn WebGL 1 to understand existing code.

WebGL 2.0 on iOS

Test WebGL 2 support here:

• https://webglreport.com/?v=2
• https://get.webgl.org/webgl2/

WebGL 2.0 : standardized extensions

    Depth Textures (WEBGL_depth_texture)
    Floating Point Textures (OES_texture_float/OES_texture_float_linear)
    Half Floating Point Textures (OES_texture_half_float/OES_texture_half_float_linear)
    Vertex Array Objects (OES_vertex_array_object)
    Standard Derivatives (OES_standard_derivatives)
    Instanced Drawing (ANGLE_instanced_arrays)
    UNSIGNED_INT indices (OES_element_index_uint)
    Setting gl_FragDepth (EXT_frag_depth)
    Blend Equation MIN/MAX (EXT_blend_minmax)
    Direct texture LOD access (EXT_shader_texture_lod)
    Multiple Draw Buffers (WEBGL_draw_buffers)
    Texture access in vertex shaders

when WebGL 2.0 is not supported, we can get close to it using WebGL 1.0 + these extensions!

WebGL 1.0.1

WebGL 1.0.1 == WebL 1.0 + omnipresent extensions

ANGLE_instanced_arrays
EXT_blend_minmax
OES_element_index_uint
OES_standard_derivatives
OES_vertex_array_object // use it!
WEBGL_debug_renderer_info
WEBGL_lose_context

always available, use them!

WebGL 1.0.2

WebGL 1.0.2 == WebL 1.0.1 + omnipresent extensions (since 2021)

EXT_texture_filter_anisotropic
OES_texture_float
OES_texture_float_linear
OES_texture_half_float
OES_texture_half_float_linear
WEBGL_depth_texture 

always available, use them!

WebGL 1.0.3

WebGL 1.0.3 == WebL 1.0.2 + omnipresent extensions (since 2022):

EXT_shader_texture_lod
EXT_sRGB
EXT_frag_depth

• but one very useful extension is not supported on Android

WEBGL_draw_buffers 

check availability before use

WebGL 2.0 extensions

WebGL 2.0 omnipresent extensions since 2022:

EXT_texture_filter_anisotropic
OES_texture_float_linear
WEBGL_debug_renderer_info
WEBGL_lose_context 

• but one fails on Safari (EXT_float_blend ?): problem for GPGPU

EXT_color_buffer_float

check availability before use

WebGPU (reminders)

• low-level API, fast, promising, introduced by Apple
• close to Metal, Vulkan and DirectX 12
  • read Metal docs to understand the concepts
• new shader language: WGSL
  • text format, gets compiled as SPIR-V (Vulkan)
• version 1.0 released on Chrome in april 2023
• official W3C spec, demos, minimalistic code sample

probably the future Web 3D API, keep an eye on it!

Conclusion: which API should I use?

• Start with WebGL 1.0, many exampled (13-year-old API!)
  • to understand the concepts
    • state machine, pipeline, buffers, shaders
    • see OpenGL course!
  • and how WebGL interacts with a web page
    • see HTML / JavaScript / CSS course
• Check new features introduced in WebGL 2.0
• Use high-level APIs: THREE.js, Babylon, A-Frame etc.
  • for a smooth transition to WebGPU!

Concepts > Syntax

• APIs evolve
• GPUs change too
• But all modern APIs and GPUs have a lot in common
• Common, transposable concepts are more important than syntax and APIs

understand how GPUs work for maximum performance

transpose your knowledge easily to other APIs, OSes or architectures

CPU

GPU

Goal

Send as much data as possible to the GPU, for fast processing

• "upload" (CPU GPU) is slow
  • group data into buffers before transfer
• GPU processing is very fast
  • using shaders
    • working in parallel
    • simple instructions
    • compiled in native low-level GPU code

Constraints

• Rendering is fast but "download" (CPU GPU) VERY slow
• Buffers are not flexible for dynamic data
• Arrays must be converted to textures (for GPGPU)
  • conversion takes time especially for dynamic data
  • possible loss of accuracy
• Shaders are complex to write
  • pixels are isolated, processed in parallel, independently
  • instructions are limited
  • optimizing and debugging is not trivial!

WebGL is a state machine too!

• data preparation
  • format, type, buffers...
• global state preparation
  • color, blending...
• rendering
  • send to GPU for shader processing

Shaders are essential

They allow to unleash the power of the GPU

• Shader code is fast
  • thousands of specialized cores inside a GPU!
  • once the data and the compiled code have been sent to the GPU, the performance is the same regardless of thelanguage or API
• Rendering is flexible
  • the rendering pipeline was fixed, not programmable before 2001
  • "rendering" has been hijacked to perform fast parallel physics and machine learning computations on the GPU (GPGPU)

Shader programming steps

• the application sends to the GPU:
  • buffers (vertices, normals, connectivity info...) and textures
  • shaders to compile and run
• vertex shaders are called once per vertex
• each primitive (point, line, triangle) is converted to fragments(rasterization)
• pixel fragment shaders are called once per fragment
  • their inputs (color, depth, normal) have been previously interpolated using the points defining the primitive!

GLSL: Shader Programming Language

• variable types

  • uniform: input, sent by the app, constant in the shader code
  • attribute: input of the vertex shader, sent by the app: data of the vertex buffer, varies per vertex
  • varying: output of the vertex shader / input of the fragment shader

• functions: C language dialect

• GLSL for WebGL 1.0, cf page 3

• GLSL pour WebGL 2.0: version OpenGL ES 3.0, cf page 8

• WebGL History

https://web.eecs.umich.edu/~sugih/courses/eecs487/lectures/20-History+ES+WebGL.pdf

• WebGL 2 Course

https://perso.univ-rennes1.fr/pierre.nerzic/IAI2/IMR2 - Synthèse d'images - CM2.pdf

• Tools for analyzing, debugging, checking and dumping WebGL

https://github.com/greggman/webgl-helpers#webgl-gl-error-checkjs