Ah one thing I need to nail before geometry shaders is better support for FBOs as I remembered I need a way to draw to mutiple textures from one shader.

(defpipeline prog-1 ((vert vert-data))
  (:vertex (setf gl-position (pos vert))
           (out (the-color :smooth) (col vert)))
  (:fragment (let ((lerp-value (/ (y gl-frag-coord) 500.0)))
               (out outputColor (mix the-color 
                                     (v! 0.2 0.2 0.2 1.0)
                                     lerp-value)))))

Currently setting the out vars of a shader stage is done with the out special form. The above code compiles to

// vertex shader
#version 330

layout(location = 0) in vec4 fk_vert_position;
layout(location = 1) in vec4 fk_vert_colour;
smooth out vec4 the_color;

void main() {
    gl_Position = fk_vert_position;
    the_color = fk_vert_colour;
}

// fragment shader
#version 330

smooth in vec4 the_color;
out vec4 outputcolor;

void main() {
    float lerp_value_4v = (gl_FragCoord.y / 500.0f);
    outputcolor = mix(the_color,vec4(0.2f,0.2f,0.2f,1.0f),lerp_value_4v);
}

Notice how the out form essentially compiles to a global var and a setf.

In the quest for lispyness, should this stay as it is? There isn’t anything wrong with setf per se, however it is worth investigating other forms to see if anything surprises us…and by us I mean me :)

Here is a version with a version of values that allows naming

(defpipeline prog-1 ((vert vert-data))
  (:vertex (values (gl-position (pos vert))
                   ((the-color :smooth) (col vert))))
  (:fragment
   (values (outputColor
            (let ((lerp-value (/ (y gl-frag-coord) 500.0)))
              (mix the-color 
                   (v! 0.2 0.2 0.2 1.0)
                   lerp-value))))))

Eh..the vertex shader is ok but this introduces an extra level of indentation that looks ugly

Ok so how about setf style? Setf allows a name-value style to set multiple variables

(setf a 1
      b 2)

So for the shaders that is…

(defpipeline prog-1 ((vert vert-data))
  (:vertex (values gl-position (pos vert)
                   (the-color :smooth) (col vert)))
  (:fragment
   (values
    outputColor
    (let ((lerp-value (/ (y gl-frag-coord) 500.0)))
      (mix the-color 
           (v! 0.2 0.2 0.2 1.0)
           lerp-value)))))

.. nope, doesn’t feel better aesthetically and now makes identifying the val and name forms more difficult.

Declare style?

(defpipeline prog-1 ((vert vert-data))
  (:vertex
   (declare (out gl-position (the-color :smooth)))
   (values (pos vert) (col vert)))
  (:fragment
   (declare (out outputColor))
   (values (let ((lerp-value (/ (y gl-frag-coord) 500.0)))
             (mix the-color 
                  (v! 0.2 0.2 0.2 1.0)
                  lerp-value)))))

..Interesting, yesterday I was looking at declare for types, so if that was the case then using this would feel natural.

Lets see what happens if the types are in declare too

(defpipeline prog-1 (vert) (:vertex (declare (vert-data vert) (out gl-position (the-color :smooth))) (values (pos vert) (col vert))) (:fragment (declare (out outputColor)) (values (let ((lerp-value (/ (y gl-frag-coord) 500.0))) (mix the-color (v! 0.2 0.2 0.2 1.0) lerp-value)))))

Haha, not much as almost everything here was already being done with inference.

There is still an issue though that is best summed up in Erik Naggum’s quote on c++. For my purposes the fact that this is about c++ is irrelevant.

C++ is philosophically and cognitively unsound as it forces a violation
of all known epistemological processes on the programmer.  as a language,
it requires you to specify in great detail what you do not know in order
to obtain the experience necessary to learn it. -- Erik Naggum

Declaring types in advanced forces the programmer to make an assertion about the nature of the code not yet written. This violates the concept of live coding as we are coding, not because we have the solution, but because we are looking for one.

Now obviously with coding in the restricted environment of the GPU types are currently a necessity so the best we can do is minimize the cognitive damage. Other than having inference everywhere we possibly can, the other way is to delay specifying the type until it is actually needed, in most cases with the arg definition.

Hmm so no progress on this but I’m glad I’ve been able to empty my head a little.

Goodnight

OK so at the end of the last post I started musing about having sensible defaults for uniforms values. The logic behind this follows:

So we have a shader ‘vert’

(defvshader vert ((position :vec4) &uniform (i :int) (loop :float))
  (let ((pos (v! (* (s~ position :xyz) 0.3) 1.0)))
    (setf gl-position (+ pos (calc-offset (+ (float i)) loop)) )))

And now I want to add a new uniform: (thing :int)

So do I add the new argument to vert, compile and it freaks out as it has no value? Or do I add it to the calls to vert first and watch it freak out as the shader doesn’t take this?

Both modes suck, and it obviously makes sense to modify the shader first, but on first compile have some default value that won’t cause the shader to panic.

This is easy for numbers, vectors and matrices as all have a concept of an identity, but what about structs?

Well all gl-structs ultimately have to be composed of types glsl understands, therefore we could have the defglstruct method generate an identity value we can use.

What about textures? Do we create a default unit texture for each sampler type? Could do. And we don’t have to worry about ‘index out of bounds’ as by default indexes just wrap for textures. (this is not true for texelFetch, the result is undefined, but shouldnt crash so will but enough time to provide real values)

according to here: http://stackoverflow.com/questions/6354208/glgentextures-is-there-a-limit-to-the-number-of-textures

"The only limit of glGenTextures is given by the bit width of the texture name (GLint), which is 32 bit;"

Which means one (1x1..n) texture for each of the 18 sampler types shouldn’t be an issue.

Now what about uniform buffers? I guess I will want something similar to gl-structs, but as I haven’t implemented UBOs yet I won’t worry about it until I am.

Right, back soon!

Ciao

After having merged in all the nice changes to foreign arrays I have decided to start looking at what is probably the biggest hole in the glsl compiler right now, Geometry Shaders.

I’m beavering away on some refactoring and code cleanup to make this tractable, which is going rather well. There should be no major changes to the compiler needed as I added all the geometry shader specific functions ages ago. I think it will mainly be glsl ‘out’ variable type transformations. I will also need to be able to specify the geometry type (tris, points, lines) in the context as that logic will be needed for proving correctness in the geometry stage.

I am not too fussed about adding Tessellation shaders urgently, mainly because I haven’t learned how to use them yet.

I am also musing over the idea of supporting the use of ‘declare’ to specify shader argument types and the context. So that the following

(defvshader test ((verts pnc) &uniform (i :int) (origin :vec3)
                  (cam-to-world :mat4) &context :version 330)
  ...shader code here)

Would instead look like:

(defvshader test (verts &uniform i origin cam-to-world)
  (declare (pnc verts) (:int i) (:vec3 origin)
           (:mat4 cam-to-world) (context :version 330))
  ...shader code here)

Which is more lispy…but as types are mandatory I’m not sure how I feel about this. Does this also mean I specify ‘out’ variables here?

** NOTE: At the start of this post I knew what I was going to blog about, but now I am off on a mental wander. So sorry for the round about nature of the rest of the post :) **

It does make the main signature clearer, especially where array types are concerned.

(i (:int 3))

The above is the current signature for an array of 3 ints, get a few of these in the argument list and it gets messy.

But this also means I will have to do the same for labels…

(labels ((test ((i :int) (y (:int 3)))
           (+ i (aref y 0))))
  (test 1 some-int-array))

..becomes..

(labels ((test (i y)
           (declare (:int i) ((:int 3) y))
           (+ i (aref y 0))))
  (test 1 some-int-array))

Hmm, not too bad, the first example is a jumble… how about ‘let’ forms?

(let ((x 1)
      ((y :int) 2)
      ((z (:int 3)) some-array))
  (some-func x y z))

Notice how the types are optional as they can be inferred, so in declare style

(let ((x 1)
      (y 2)
      (z some-array))
  (declare (:int y) ((:int 3) some-array))
  (some-func x y z))

Ok now in this case it is a BIG improvement to the actual readability of the declarations themselves.

The big worry for me though is giving you two places to look for one piece of information when reading the code. What you want to know is “What is x” and you have to look at for it’s declaration (so you can know it’s scope) and the declare (to know it’s type).

The other possibility is to be super un-common-lispy and use some special reader syntax that is ONLY valid in bindings.

(labels ((test ([i :int] [y (:int 3)])
           (+ i (aref y 0))))
  (test 1 some-int-array))

(let ((x 1)
      ([y :int] 2)
      ([z (:int 3)] some-array))
  (some-func x y z))

Ugh…even writing it down feels wrong. Nah, scrap this idea. I think the declare style is growing on me but I will need to give this some time. I’d appreciate any ideas on this one.

Ciao

p.s. But wait, one other advantage of declare form, it allows us to have default values for uniforms args. Hmm, this quite possibly means more state changes, as every unspecified uniform must take the defaut value, whereas currently uniforms are memoized. Ok so bad idea. I should probably have the concept of sane defaults though. I should move this to a new post :)

First some code!:

> (setf c (make-c-array #(1 2 3 4)))
#<C-ARRAY :element-type :UBYTE :dimensions (4)>

> (setf g (make-gpu-array #(1 2 3 4)))
#<GPU-ARRAY :element-type :UBYTE :dimensions (4) :backed-by :BUFFER>

> (setf g (make-texture #(1 2 3 4)))
#<GL-TEXTURE-1D (4)>

> (texref *)
#<GPU-ARRAY :element-type :R8 :dimensions (4) :backed-by :TEXTURE>

I have tweaked the make-*-array and make-*-texture commands so that, if you only pass in a lisp data structure, cepl will try and find a the smallest gl type that can be used across the whole data-set.

For example:

(make-c-array #(1 2 3 4)) ;; <- element type will be :ubyte

(make-c-array #(1 -2 3 4)) ;; <- element type will be :byte

(make-c-array #(1 -2 200 4)) ;; <- element type will be :int

(make-c-array #(1 -2 200 4.0)) ;; <- element type will be :float

(make-c-array #(1 -2 200 4.0d0)) ;; <- element type will be :double

This is one of those nice cases where you get to make repl exploration more fun with no cost to speed. The reason for this is no high level code will be using this inference as it would be stunningly stupid to pay that cost for no reason. All file loading (which is the most common case) will have the type name anyway so it can just be passed in as usual.

Little details, but pleasant ones

p.s. This is in a branch for now as I am hunting down some issues I have introduced while modifying the indexing functions

http://osxdaily.com/2012/05/22/install-wget-mac-os-x/

Go check it out, very useful.

I have been hoilding off pushing the new features to Cepl recently as I have one bug in defglstruct that really is confusing me.

At first it seems very simple, it is complaining that certain functions (A) dont exist with compiling functions (B). Now (A) do exist by the time (B) is called so it doesnt cause any immediate problems, but the compiler wanrings are very annoying.

The weird bit is that (A) do exist at that time. For example if I expand the defglstruct macro and then run the resulting code it works with no warnings.

There is something I’m just missing but it is the reason I havent merged into master yet.

I may give this some more attention tonight.

Ciao

Last night I was musing over rendering graphs into textures and I was interested in how to write just a couple of pixels to the texture without reuploading the image. Now I can use gl’s ‘copy sub image’ functions but looking through my copy I saw I hadn’t implemented buffer backed textures; this gave me a great excuse to finally get it done!

In cepl buffer backed arrays would more accurately be named array-backed-textures as we already have the gpu-array abstraction over buffers. This works in my favour though as currently the abstraction for textures in cepl is as follows:

• A texture is a structure that holds a number of images. You use texref (like aref) to get at one of the images.
• Images are just arrays so they are exposed as texture backed gpu-arrays.

So calling texref on a regular texture gives you a texture-backed gpu-array and calling texref on a buffer-backed texture give you a buffer-backed gpu-array.

So to get change one pixel of a texture to the color red we can do this:

(with-gpu-array-as-c-array ((texref texture) :tempname arr)
  (setf (aref-c arr 10 10) (v! 1 0 0)))

I have been coding a lot of c++, java and c# recently and so coming back to lisp and doing things like the above just makes me so damn happy! I love this language.

Hi folks, having recently got married (yay!) and got a flat (another yay!) I have been getting back into lisp coding again.

The latest addition to the Varjo compiler is support for ‘values’ and ‘multiple-value-bind’. Now as glsl doesn’t have support for multiple value returns we have to use something else. The mechanism I have now works something like this:

• If you have a ‘values’ form within a multiple-value-bind form the multiple-value-bind form turns into a ‘let’ and the ‘values’ becomes ‘setf’s to the variables in the let.

• If you have a values form and it isnt within a multiple-value-bind then one of the following happens ** If the values form can propagate to a ‘return’ form then the function is given out-vars and they are set from the place where the values form was originally. ** If it cant reach a return then it collapses to a progn.

If a varjo function with out-vars is used within a multiple-value-bind then the multiple-value-bind becomes a let statement and the first var is set by the function return and the rest are passed as out-vars to the function.

If the function is used outside of a multiple-value-bind then the form is surrounded in a ‘let’ as the outvars still need to be set.

This results is something like the following.

VARJO> (defshader test ()
         (labels ((thing ((x :int)) (return (values x (* x 2)))))
           (multiple-value-bind (x y) (thing 1)
             (+ x y)))))

;; Gets compiled to

#version 330

int thing_26v(int x_23v, out int return1);

int thing_26v(int x_23v, out int return1) {
    return0 = x_23v;
    return1 = (x_23v * 2);
    return return0;
}

void main() {
    int a_27v0;
    int a_27v1;
    a_27v0 = thing_26v(1,a_27v1);
    (a_27v0 + a_27v1);
}

I still have a few bugs to iron out but it is very nearly there. One step closer to lisp!

I tried adding an existing android project today and got this error from Eclipse.

“<name of project> overlaps the location of another project..”

It was reported as a project desciption problem. Turns out eclipse doesnt like importing projects from the direcotry it has designated as the workspace. Move the project outside that folder (or change the workspace directory) and try again.

What a pain in the arse!