(Antialiasing 3d)
 
(10 intermediate revisions by the same user not shown)
Line 1: Line 1:
 
Question here below is simple: how to produce moving images with colored dots. The idea has been experimented in several technical and visual forms, all based on the manipulation of simple basic entities.
 
Question here below is simple: how to produce moving images with colored dots. The idea has been experimented in several technical and visual forms, all based on the manipulation of simple basic entities.
 +
 +
[[File:PelleasetMelisande-NicolasDescoteaux-OperaDeBordeaux-designvideo-9.jpg|link=WorkUnit:Particles#Pélléas et Mélissande|640px]]
  
 
Researches related to particles, in different context:
 
Researches related to particles, in different context:
Line 5: Line 7:
 
* colliding with a 3d model, researches of 2013
 
* colliding with a 3d model, researches of 2013
 
* in relation with a vector field, for Pelléas et Mélisande opéra.
 
* in relation with a vector field, for Pelléas et Mélisande opéra.
 +
 +
== Accumulation management ==
 +
 +
FBO<ref>Framebuffer Object - https://www.khronos.org/opengl/wiki/Framebuffer_Object</ref> pipeline that:
 +
 +
* control the accumulation of the particles in time by making them disappear gradually (fade2black)
 +
* transform the luminosity of pixels into alpha, darker => more transparent (lum2alpha)
 +
 +
Passes, by columns:
 +
 +
# current frame
 +
# fade2black shader<ref>Shader - https://www.khronos.org/opengl/wiki/Shader</ref> and accumulation of the current on top of previous ones
 +
# lum2alpha shader, this is the output frame, shown on checkboard, black and green background
 +
# delta between second and third pass, showing the color loss during the lum2alpha pass
 +
 +
[[File:Accumulation alpha shaders.png|600px]]
 +
 +
=== shaders ===
 +
 +
All shaders below are pixels shaders. The vertex shader associated is ultra basic, and i'm not even sure you need one... (openframeworks binding)
 +
 +
'''vertex shader'''
 +
 +
#version 120
 +
void main() {
 +
    gl_Position = ftransform();
 +
}
 +
 +
'''fade2black'''
 +
 +
#version 120
 +
uniform sampler2DRect tex0;
 +
uniform float decay_factor;
 +
uniform float decay_mult;
 +
const float PI = 3.14159265358979323846;
 +
const float HALF_PI = 1.57079632679489661923;
 +
void main() {
 +
    vec4 c = texture2DRect(tex0, gl_FragCoord.xy);
 +
    float dfi = 1.0 - decay_factor;
 +
    c.r -=  ( 1 - ( ( 1 + sin( -HALF_PI + c.r * PI ) ) * 0.5 ) ) * decay_factor;
 +
    c.g -=  ( 1 - ( ( 1 + sin( -HALF_PI + c.g * PI ) ) * 0.5 ) ) * decay_factor;
 +
    c.b -=  ( 1 - ( ( 1 + sin( -HALF_PI + c.b * PI ) ) * 0.5 ) ) * decay_factor;
 +
    c.r *= decay_mult;
 +
    c.g *= decay_mult;
 +
    c.b *= decay_mult;
 +
    gl_FragColor = c;
 +
}
 +
 +
'''lum2alpha'''
 +
 +
#version 120
 +
// http://www.odelama.com/data-analysis/How-to-Compute-RGB-Image-Standard-Deviation-from-Channels-Statistics/
 +
// luminosity per channel in sRGB
 +
//  Y′ = 0.2126⋅Red′ + 0.7152⋅Green′ + 0.0722⋅Blue′
 +
//const float lum_red = 0.2126;
 +
//const float lum_green = 0.7152;
 +
//const float lum_blue = 0.0722;
 +
// adobe RGB 1998
 +
// Y′ = 0.2974⋅Red′ + 0.6273⋅Green′ + 0.0753⋅Blue′
 +
const float lum_red = 0.2974;
 +
const float lum_green = 0.6273;
 +
const float lum_blue = 0.0753;
 +
uniform sampler2DRect tex0;
 +
uniform float alpha_min;
 +
uniform float alpha_pow;
 +
 +
void main() {
 +
    vec4 c = texture2DRect(tex0, gl_FragCoord.xy);
 +
    float darkness = 1 - min(1, c.r * lum_red + c.g * lum_green + c.b * lum_blue);
 +
    float luminosity = (1 - pow(darkness, alpha_pow));
 +
    c.a = alpha_min + (1 - alpha_min) * luminosity;
 +
    gl_FragColor = c;
 +
    // debug
 +
    // gl_FragColor = vec4( c.a, c.a, c.a, 1 );
 +
 +
}
 +
 +
'''delta'''
 +
 +
#version 120
 +
uniform sampler2DRect tex0;
 +
uniform sampler2DRect tex_alpha;
 +
const float amplification = 1;
 +
 +
void main() {
 +
    vec4 c0 = texture2DRect(tex0, gl_FragCoord.xy);
 +
    vec4 c1 = texture2DRect(tex_alpha, gl_FragCoord.xy);
 +
    gl_FragColor = vec4(
 +
            abs(c0.r - c1.r * c1.a) * amplification,
 +
            abs(c0.g - c1.g * c1.a) * amplification,
 +
            abs(c0.b - c1.b * c1.a) * amplification,
 +
            1);
 +
}
  
 
== Vector field as a controller ==  
 
== Vector field as a controller ==  
Line 12: Line 107:
 
=== Antialiasing 3d ===
 
=== Antialiasing 3d ===
  
More precisely: how to avoid anti-aliasing<ref>Aliasing - https://en.wikipedia.org/wiki/Aliasing</ref> by using sub-sampling<ref>What is the difference between Binning and sub-sampling in Image Signal Processing? - https://stackoverflow.com/questions/31529379/what-is-the-difference-between-binning-and-sub-sampling-in-image-signal-processi#31529981</ref>, implying identification of a method to go from discrete to continuous<ref>Probability Distributions: Discrete vs. Continuous - http://stattrek.com/probability-distributions/discrete-continuous.aspx</ref> influence of the vector field. Below with human words.
+
More precisely: aliasing<ref>Aliasing - https://en.wikipedia.org/wiki/Aliasing</ref> the vector field<ref>Vector field - https://en.wikipedia.org/wiki/Vector_field</ref>'s influence on the particles by using sub-sampling<ref>What is the difference between Binning and sub-sampling in Image Signal Processing? - https://stackoverflow.com/questions/31529379/what-is-the-difference-between-binning-and-sub-sampling-in-image-signal-processi#31529981</ref>, implying identification of a method to go from discrete to continuous<ref>Probability Distributions: Discrete vs. Continuous - http://stattrek.com/probability-distributions/discrete-continuous.aspx</ref>. Below with human words.
  
Until this point, particles were influenced by only one cell currently, implying brutal trajectory's modification when the forces are different from one cell to the other...
+
Until this point, particles were influenced by only one cell at the time, implying brutal trajectory's modification when the forces are different from one cell to the other...
  
It took me some time, but the issue is similar to an exercise I gave to the students at [http://arts2.be arts²]: how to create a gradient between 4 points. In the vector field, the gradient has one more dimension but has the same behavior.
+
It tooks me some a bit of time, but the issue is similar to an exercise I gave to the students at [http://arts2.be arts²]: how to create a gradient between 4 points. In the vector field, the gradient has one more dimension (and therefore uses 8 points) but has the same behavior.
  
Any particle is always between 4 cells, let's call this a '''cluster'''. This cluster does not have to contains the totality of the cells, it must just connect the 4 centers. Therefore, a cluster has exactly the same size has a cell in the vector field, and its position is offset by an half cell. Too simple to see it at the first glance, because it requires to look at the space between center, and not the frontier of the cells (typical case where out-of-the-box thinking is required!).
+
Any particle is always between 8 cells. Let's call this block of 8 cells a '''cluster'''. This cluster does not have to contains the totality of the cells, it must just connect the 8 centers. Therefore, a cluster has exactly the same size has a cell in the vector field, and its position is offset by an half cell. Too simple to see it at the first glance, because it requires to look at the space between center, and not the frontier of the cells (typical case where out-of-the-box thinking is required!).
  
 
[[File:Particles-cells-interpolation 01.jpg|400px]] [[File:Particles-cells-interpolation 02.jpg|400px]]
 
[[File:Particles-cells-interpolation 01.jpg|400px]] [[File:Particles-cells-interpolation 02.jpg|400px]]
Line 35: Line 130:
  
 
== History ==
 
== History ==
 +
 +
=== Pélléas et Mélissande ===
 +
 +
[[File:PelleasetMelisande-NicolasDescoteaux-OperaDeBordeaux-designvideo-9.jpg|640px]]
 +
 +
Création à l'Opéra de Bordeaux, Video Design / Video scenography by [http://www.thomasisrael.be Thomas Israël].
  
 
=== Fragments #43-44 ===
 
=== Fragments #43-44 ===
Line 57: Line 158:
  
 
<references/>
 
<references/>
 +
 +
[[category:openframeworks.cc]]
 +
[[category:shaders]]
 +
[[category:generative]]
 +
[[category:simulation]]
 +
[[category:3D]]
 +
[[category:algorithm]]

Latest revision as of 17:15, 11 March 2018

Question here below is simple: how to produce moving images with colored dots. The idea has been experimented in several technical and visual forms, all based on the manipulation of simple basic entities.

PelleasetMelisande-NicolasDescoteaux-OperaDeBordeaux-designvideo-9.jpg

Researches related to particles, in different context:

  • colliding with a 3d model, researches of 2013
  • in relation with a vector field, for Pelléas et Mélisande opéra.

Accumulation management

FBO[1] pipeline that:

  • control the accumulation of the particles in time by making them disappear gradually (fade2black)
  • transform the luminosity of pixels into alpha, darker => more transparent (lum2alpha)

Passes, by columns:

  1. current frame
  2. fade2black shader[2] and accumulation of the current on top of previous ones
  3. lum2alpha shader, this is the output frame, shown on checkboard, black and green background
  4. delta between second and third pass, showing the color loss during the lum2alpha pass

Accumulation alpha shaders.png

shaders

All shaders below are pixels shaders. The vertex shader associated is ultra basic, and i'm not even sure you need one... (openframeworks binding)

vertex shader

#version 120
void main() {
   gl_Position = ftransform();
}

fade2black

#version 120
uniform sampler2DRect tex0;
uniform float decay_factor;
uniform float decay_mult;
const float PI = 3.14159265358979323846;
const float HALF_PI = 1.57079632679489661923;
void main() {
   vec4 c = texture2DRect(tex0, gl_FragCoord.xy);
   float dfi = 1.0 - decay_factor;
   c.r -=  ( 1 - ( ( 1 + sin( -HALF_PI + c.r * PI ) ) * 0.5 ) ) * decay_factor;
   c.g -=  ( 1 - ( ( 1 + sin( -HALF_PI + c.g * PI ) ) * 0.5 ) ) * decay_factor;
   c.b -=  ( 1 - ( ( 1 + sin( -HALF_PI + c.b * PI ) ) * 0.5 ) ) * decay_factor;
   c.r *= decay_mult;
   c.g *= decay_mult;
   c.b *= decay_mult;
   gl_FragColor = c;
}

lum2alpha

#version 120
// http://www.odelama.com/data-analysis/How-to-Compute-RGB-Image-Standard-Deviation-from-Channels-Statistics/
// luminosity per channel in sRGB
//  Y′ = 0.2126⋅Red′ + 0.7152⋅Green′ + 0.0722⋅Blue′
//const float lum_red = 0.2126;
//const float lum_green = 0.7152;
//const float lum_blue = 0.0722;
// adobe RGB 1998
// Y′ = 0.2974⋅Red′ + 0.6273⋅Green′ + 0.0753⋅Blue′
const float lum_red = 0.2974;
const float lum_green = 0.6273;
const float lum_blue = 0.0753;
uniform sampler2DRect tex0;
uniform float alpha_min;
uniform float alpha_pow;

void main() {
   vec4 c = texture2DRect(tex0, gl_FragCoord.xy);
   float darkness = 1 - min(1, c.r * lum_red + c.g * lum_green + c.b * lum_blue);
   float luminosity = (1 - pow(darkness, alpha_pow));
   c.a = alpha_min + (1 - alpha_min) * luminosity;
   gl_FragColor = c;
   // debug
   // gl_FragColor = vec4( c.a, c.a, c.a, 1 );

}

delta

#version 120
uniform sampler2DRect tex0;
uniform sampler2DRect tex_alpha;
const float amplification = 1;

void main() {
   vec4 c0 = texture2DRect(tex0, gl_FragCoord.xy);
   vec4 c1 = texture2DRect(tex_alpha, gl_FragCoord.xy);
   gl_FragColor = vec4(
           abs(c0.r - c1.r * c1.a) * amplification,
           abs(c0.g - c1.g * c1.a) * amplification,
           abs(c0.b - c1.b * c1.a) * amplification,
           1);
}

Vector field as a controller

started on the 15 december 2017

Antialiasing 3d

More precisely: aliasing[3] the vector field[4]'s influence on the particles by using sub-sampling[5], implying identification of a method to go from discrete to continuous[6]. Below with human words.

Until this point, particles were influenced by only one cell at the time, implying brutal trajectory's modification when the forces are different from one cell to the other...

It tooks me some a bit of time, but the issue is similar to an exercise I gave to the students at arts²: how to create a gradient between 4 points. In the vector field, the gradient has one more dimension (and therefore uses 8 points) but has the same behavior.

Any particle is always between 8 cells. Let's call this block of 8 cells a cluster. This cluster does not have to contains the totality of the cells, it must just connect the 8 centers. Therefore, a cluster has exactly the same size has a cell in the vector field, and its position is offset by an half cell. Too simple to see it at the first glance, because it requires to look at the space between center, and not the frontier of the cells (typical case where out-of-the-box thinking is required!).

Particles-cells-interpolation 01.jpg Particles-cells-interpolation 02.jpg

History

Pélléas et Mélissande

PelleasetMelisande-NicolasDescoteaux-OperaDeBordeaux-designvideo-9.jpg

Création à l'Opéra de Bordeaux, Video Design / Video scenography by Thomas Israël.

Fragments #43-44

Fragments#43-44 blends musical improvisation, cinematic soundscapes as well as visuals generated on the fly by an intriguing new body language, src: http://fragments4344.uranium.be/

Colors’ flood on landscape

Demo, september 4, 2013

Resources

References

  1. Framebuffer Object - https://www.khronos.org/opengl/wiki/Framebuffer_Object
  2. Shader - https://www.khronos.org/opengl/wiki/Shader
  3. Aliasing - https://en.wikipedia.org/wiki/Aliasing
  4. Vector field - https://en.wikipedia.org/wiki/Vector_field
  5. What is the difference between Binning and sub-sampling in Image Signal Processing? - https://stackoverflow.com/questions/31529379/what-is-the-difference-between-binning-and-sub-sampling-in-image-signal-processi#31529981
  6. Probability Distributions: Discrete vs. Continuous - http://stattrek.com/probability-distributions/discrete-continuous.aspx

online identity ∋ [ social ∋ [mastodon♥, twitter®, facebook®, diaspora, linkedin®] ∥ repos ∋ [github®, gitlab♥, bitbucket®, sourceforge] ∥ media ∋ [itch.io®, vimeo®, peertube♥, twitch.tv®, tumblr®] ∥ communities ∋ [godotengine♥, openprocessing, stackoverflow, threejs]]