From the Burrow

Soon there will be a new Stage

2014-12-02 23:50:22 +0000

After having merged in all the nice changes to foreign arrays I have decided to start looking at what is probably the biggest hole in the glsl compiler right now, Geometry Shaders.

I’m beavering away on some refactoring and code cleanup to make this tractable, which is going rather well. There should be no major changes to the compiler needed as I added all the geometry shader specific functions ages ago. I think it will mainly be glsl ‘out’ variable type transformations. I will also need to be able to specify the geometry type (tris, points, lines) in the context as that logic will be needed for proving correctness in the geometry stage.

I am not too fussed about adding Tessellation shaders urgently, mainly because I haven’t learned how to use them yet.

I am also musing over the idea of supporting the use of ‘declare’ to specify shader argument types and the context. So that the following

(defvshader test ((verts pnc) &uniform (i :int) (origin :vec3)
                  (cam-to-world :mat4) &context :version 330)
  ...shader code here)

Would instead look like:

(defvshader test (verts &uniform i origin cam-to-world)
  (declare (pnc verts) (:int i) (:vec3 origin)
           (:mat4 cam-to-world) (context :version 330))
  ...shader code here)

Which is more lispy…but as types are mandatory I’m not sure how I feel about this. Does this also mean I specify ‘out’ variables here?

** NOTE: At the start of this post I knew what I was going to blog about, but now I am off on a mental wander. So sorry for the round about nature of the rest of the post :) **

It does make the main signature clearer, especially where array types are concerned.

(i (:int 3))

The above is the current signature for an array of 3 ints, get a few of these in the argument list and it gets messy.

But this also means I will have to do the same for labels…

(labels ((test ((i :int) (y (:int 3)))
           (+ i (aref y 0))))
  (test 1 some-int-array))


(labels ((test (i y)
           (declare (:int i) ((:int 3) y))
           (+ i (aref y 0))))
  (test 1 some-int-array))

Hmm, not too bad, the first example is a jumble… how about ‘let’ forms?

(let ((x 1)
      ((y :int) 2)
      ((z (:int 3)) some-array))
  (some-func x y z))

Notice how the types are optional as they can be inferred, so in declare style

(let ((x 1)
      (y 2)
      (z some-array))
  (declare (:int y) ((:int 3) some-array))
  (some-func x y z))

Ok now in this case it is a BIG improvement to the actual readability of the declarations themselves.

The big worry for me though is giving you two places to look for one piece of information when reading the code. What you want to know is “What is x” and you have to look at for it’s declaration (so you can know it’s scope) and the declare (to know it’s type).

The other possibility is to be super un-common-lispy and use some special reader syntax that is ONLY valid in bindings.

(labels ((test ([i :int] [y (:int 3)])
           (+ i (aref y 0))))
  (test 1 some-int-array))

(let ((x 1)
      ([y :int] 2)
      ([z (:int 3)] some-array))
  (some-func x y z))

Ugh…even writing it down feels wrong. Nah, scrap this idea. I think the declare style is growing on me but I will need to give this some time. I’d appreciate any ideas on this one.


p.s. But wait, one other advantage of declare form, it allows us to have default values for uniforms args. Hmm, this quite possibly means more state changes, as every unspecified uniform must take the defaut value, whereas currently uniforms are memoized. Ok so bad idea. I should probably have the concept of sane defaults though. I should move this to a new post :)

It's Just Pretty

2014-11-25 00:21:00 +0000

First some code!:

> (setf c (make-c-array #(1 2 3 4)))
#<C-ARRAY :element-type :UBYTE :dimensions (4)>

> (setf g (make-gpu-array #(1 2 3 4)))
#<GPU-ARRAY :element-type :UBYTE :dimensions (4) :backed-by :BUFFER>

> (setf g (make-texture #(1 2 3 4)))
#<GL-TEXTURE-1D (4)>

> (texref *)
#<GPU-ARRAY :element-type :R8 :dimensions (4) :backed-by :TEXTURE>

I have tweaked the make-*-array and make-*-texture commands so that, if you only pass in a lisp data structure, cepl will try and find a the smallest gl type that can be used across the whole data-set.

For example:

(make-c-array #(1 2 3 4)) ;; <- element type will be :ubyte

(make-c-array #(1 -2 3 4)) ;; <- element type will be :byte

(make-c-array #(1 -2 200 4)) ;; <- element type will be :int

(make-c-array #(1 -2 200 4.0)) ;; <- element type will be :float

(make-c-array #(1 -2 200 4.0d0)) ;; <- element type will be :double

This is one of those nice cases where you get to make repl exploration more fun with no cost to speed. The reason for this is no high level code will be using this inference as it would be stunningly stupid to pay that cost for no reason. All file loading (which is the most common case) will have the type name anyway so it can just be passed in as usual.

Little details, but pleasant ones

p.s. This is in a branch for now as I am hunting down some issues I have introduced while modifying the indexing functions

Install wget on osx without MacPorts

2014-09-23 08:02:56 +0000

Go check it out, very useful.

Delays in Pushing Code

2014-07-28 09:55:06 +0000

I have been hoilding off pushing the new features to Cepl recently as I have one bug in defglstruct that really is confusing me.

At first it seems very simple, it is complaining that certain functions (A) dont exist with compiling functions (B). Now (A) do exist by the time (B) is called so it doesnt cause any immediate problems, but the compiler wanrings are very annoying.

The weird bit is that (A) do exist at that time. For example if I expand the defglstruct macro and then run the resulting code it works with no warnings.

There is something I’m just missing but it is the reason I havent merged into master yet.

I may give this some more attention tonight.


Buffer Backed Textures

2014-07-28 09:26:48 +0000

Last night I was musing over rendering graphs into textures and I was interested in how to write just a couple of pixels to the texture without reuploading the image. Now I can use gl’s ‘copy sub image’ functions but looking through my copy I saw I hadn’t implemented buffer backed textures; this gave me a great excuse to finally get it done!

In cepl buffer backed arrays would more accurately be named array-backed-textures as we already have the gpu-array abstraction over buffers. This works in my favour though as currently the abstraction for textures in cepl is as follows:

  • A texture is a structure that holds a number of images. You use texref (like aref) to get at one of the images.
  • Images are just arrays so they are exposed as texture backed gpu-arrays.

So calling texref on a regular texture gives you a texture-backed gpu-array and calling texref on a buffer-backed texture give you a buffer-backed gpu-array.

So to get change one pixel of a texture to the color red we can do this:

(with-gpu-array-as-c-array ((texref texture) :tempname arr)
  (setf (aref-c arr 10 10) (v! 1 0 0)))

I have been coding a lot of c++, java and c# recently and so coming back to lisp and doing things like the above just makes me so damn happy! I love this language.

Varjo and Multiple Value Return

2014-07-24 15:46:00 +0000

Hi folks, having recently got married (yay!) and got a flat (another yay!) I have been getting back into lisp coding again.

The latest addition to the Varjo compiler is support for ‘values’ and ‘multiple-value-bind’. Now as glsl doesn’t have support for multiple value returns we have to use something else. The mechanism I have now works something like this:

  • If you have a ‘values’ form within a multiple-value-bind form the multiple-value-bind form turns into a ‘let’ and the ‘values’ becomes ‘setf’s to the variables in the let.

  • If you have a values form and it isnt within a multiple-value-bind then one of the following happens ** If the values form can propagate to a ‘return’ form then the function is given out-vars and they are set from the place where the values form was originally. ** If it cant reach a return then it collapses to a progn.

If a varjo function with out-vars is used within a multiple-value-bind then the multiple-value-bind becomes a let statement and the first var is set by the function return and the rest are passed as out-vars to the function.

If the function is used outside of a multiple-value-bind then the form is surrounded in a ‘let’ as the outvars still need to be set.

This results is something like the following.

VARJO> (defshader test ()
         (labels ((thing ((x :int)) (return (values x (* x 2)))))
           (multiple-value-bind (x y) (thing 1)
             (+ x y)))))

;; Gets compiled to

#version 330

int thing_26v(int x_23v, out int return1);

int thing_26v(int x_23v, out int return1) {
    return0 = x_23v;
    return1 = (x_23v * 2);
    return return0;

void main() {
    int a_27v0;
    int a_27v1;
    a_27v0 = thing_26v(1,a_27v1);
    (a_27v0 + a_27v1);

I still have a few bugs to iron out but it is very nearly there. One step closer to lisp!

Eclipse project overlaps error

2014-05-05 14:02:42 +0000

I tried adding an existing android project today and got this error from Eclipse.

“<name of project> overlaps the location of another project..”

It was reported as a project desciption problem. Turns out eclipse doesnt like importing projects from the direcotry it has designated as the workspace. Move the project outside that folder (or change the workspace directory) and try again.

What a pain in the arse!

Android Ndk Build Error Multiple Targets

2014-05-05 13:57:56 +0000

Had this error today:

* multiple target patterns. Stop.

(Turns out I needed to delete the obj file from the android project)[]

This post is mainly to help me remember this but hopefully it helps someone else too.

Cepl on OSX

2014-04-15 17:29:22 +0000

This has been a massive pain in the arse and still isn’t working in the way I would like it but there is some progress so it is worth testing.

First off, while I love sbcl, I cannot get sdl2 to create a fully working window. I get a window with no border or one which does not receive input events.

So instead I tried clozure (ccl) and have had some success.

First cepl:repl now wraps it’s contents in an sdl2:in-main-thread form. So you can continue to use this as usual.

Next you need to wrap the code that contains your main loop in an ‘in-main-thread’ form. In the case of the provided examples you can just write (sdl2:in-main-thread () (run-demo)).

This gives you an active window that is decorated and accepts events which is wonderful. The only problem so far is that #’update-swank doesn’t work and so you cant use slime’s repl while the demo is running. This is a deal breaker for me, as that ruins my workflow so I need as solution to consider using osx for any cepl development.

Hopefully a solution to this will emerge soon. I then really need to go clean up the code surrounding this fix.

Here’s hoping

[edit] Seems the repl in the inferior-lisp buffer (or shell) is active and usable. However the repl buffer doesnt yet. Can anyone with more experience with ccl help me out here?

Fps Counter with Temporal Functions

2014-04-14 00:00:46 +0000

(let ((count 0))
  (tdefun fps ()
    (incf count)
    ((each (seconds 1)) (print count) (setf count 0))))

It’s simple but it makes me happy simply because it is written in term of time.

p.s. Oh yeah, as you may have guessed from this, temporal functions work now!