First day of November, first day of my first NanoWriMo. Traditionally this is a challenge to write a 50000 word novel in a month but I’m going to be coding along with this instead. As word/line counts are pointless in code I’m going to try do some significant work on cepl.

I’m going to aim to make a space-graph (analogous to a scenegraph) I’ll be writing this up as I go so Ι won’t burden this post with it.

Today however Ι am bug hunting by taking a rather nice shader called backscatter from shadertoy and remaking it in cepl. It was written by a good friend who tells me it’s too slow, which may be fair but it is also clearly written and pretty enough to be motivating.

writing it has already let me find a few bugs, including one in Varjo’s structs so I am off to fix that now :)

Phew, now Fuse has shipped the beta I’m not crunching so much at work so I’m back working on cepl again. (Check out Fuse btw, it’s early days but theres some kickass shit in there, I’ll be doing some posts on it in future)

I’ve not a nice branch full of changes to merge in soon in which I’ve been working on two main things:

1. Better error messages This took a lot more work than just tweaking the strings. I’ve gone though varjo (the lisp->glsl compiler) trying to find places where I can make suggestions of possible correct alternatives. The obvious example for this is variable names so if you type the wrong name varjo will make suggestions based on names that are significantly similar (I’m just using jaro-winkler-distance for this). In cepl it also involved going through the more hairy code, trying to seperate it into more sensible functions and also making sure I give more specific errors especially around the code that handles converting between texture-formats & lisp types.

2. New event system So I changed this yet again :p. The reasons for doing this have mainly been around limiting scope. I dont want unnecesarily high level abstractions in cepl core, I want ‘low enough’ abstractions that are easy to build on.

So this time the main change is that event listeners are remvoed in favour of event-nodes. Event nodes can subscribe to other event nodes to recieve their events. Any event node can also be subscribed to. Each node holds a list of weak refs to the node that are subscribed to it, and a list of strong refs to the nodes that it subscribes to. In this way we have a event graph that the garbage collector can clean up if you don’t unsubscribe explicitly.

“Why add this weak ref complexity” would be a fair question to ask, and the reason is making a balance between good practice and pragmatism. Common Lisp allows awesome levels experimentation with the repl and incremental compiling. It is inevitable that, whilst explicitly disconnecting event-nodes is the correct thing to do, managing that is a PITA when you are just trying to play with ideas. To this end we make sure we can GC away parts of the event graph that are no longer needed, letting you focus on the important stuff. This kind of thing is what I mean by ‘low level enough’, I’m not trying to bake in some glitch free reactive programming framework (though those are awesome), but I still want to make the experience good and lispy.

Less importantly, with the new changes events are now structs with read only fields.

The driving force for the above changes has been that I now have event sources other than SDL2. I have made a simple android app that sends touch events over sockets to cepl to allow more direct interaction with whatever I’m making. It’s early days but the events from the cepl remote-control are now flowing through the event system just like mouse & keyboard events so I’m pretty stoked :)

This physical hardware business makes recording youtube videos more tricky so I’m gonna go buy some extra cameras today so I can try show this stuff off.

Hopefully I’ll be back soon with a new video for you. Ciao

Not sure how long until we will be programming biology but it doesnt hurt to start knowing about the technology which will make it robust. CRISPR is an interesting topic in how it works and also with it’s reliability (relative to previous methods).

Andrew Gibiansky has writting an excellent post on CRISPR which was very easy to understand, check it out here

…the most terrifying thing code can do is run whilst you looks at some part of it so terrible that you cant understand how it even started. It’s like cutting someone in half and they walk off and try and carry on their day.

Been a bit quiet recently but yesterday I spent the day working on conditionals in cepl.

There are simply too many cases where the error is cryptic, or fails way later in the process that it should have.

Defpipeline should now through more sensible errors now and varjo will try and suggest types if you use one it doesn’t know. e.g.

VARJO> (defshader test ((x sampler2d))
         (texture x (v! 0 0)))

gives

Varjo: Varjo: Could not find the correct type for type-spec SAMPLER2D

Perhaps you meant one of these types?:
:usampler-2d
:sampler-3d
:sampler-2d-shadow
:sampler-2d-rect
:sampler-2d-ms
:sampler-2d-array
:sampler-2d
v-sampler-2d
:sampler-1d
:isampler-2d
:sampler
   [Condition of type UNKNOWN-TYPE-SPEC]

Restarts:
 0: [RETRY] Retry SLIME REPL evaluation request.
 1: [*ABORT] Return to SLIME's top level.
 2: [ABORT] abort thread (#<THREAD "repl-thread" RUNNING {1005078003}>)

I have finished cleaning up type errors for textures and will be focusing on c-arrays & fbos after this.

That should hopefully stop the cases where Ι assume that cepl is buggy rather than checking my own typos :D

I quickly hacked this together to fix fixes a frustration where trying to write #a(1 2 3) gets ‘corrected’ by paredit to #a (1 2 3)

(defvar paredit-dont-add-space-after
  '((?s ?#) (?a ?#) (?λ) (?- ?#) (?+ ?#) (?c ?#) (?o ?#) (?p ?#) (?r ?#) (?s ?#) (?x ?#)))

(add-hook 'paredit-mode-hook
          (lambda ()
            (setq paredit-space-for-delimiter-predicates
                  (list (lambda (endp delimiter)
                          (if (not endp)
                              (save-excursion
                                (let ((1-back (char-before))
                                      (2-back (char-before (- (point) 1))))
                                  (null
                                   (cl-loop for (1b 2b) in paredit-dont-add-space-after thereis
                                            (and 1-back (char-equal 1b 1-back)
                                                 (if 2b
                                                     (when 2-back
                                                       (char-equal 2b 2-back))
                                                   t))))))
                            t))))))

TLDR: Look Here

There was one thing during making my glsl compiler that was terrible; One thing worse than any of the bugs I have hammered out so far.

The GLSL Spec

The spec is available either as pdf or html. It is designed for humans to read but I needed my program to know about every function in glsl. To that end I spent days processing the contents of the pdf, and some other open code, in order to get a list of all the functions with their return & argument types.

Since then I have wanted to update all the functions with information on which versions they are available from, but this has seemed a daunting undertaking.

Just the other day however, some people have put together a new gl documentation site http://docs.gl, whilst the information itself still is geared to be human readable it is at least better laid out and stored in a github repo.

I took a few hours over the weekend to knock up a script that would extract the basic info I need from the glsl spec. Potentially I could do the opengl one too, but for now that is not urgent. You can find the results here.

The data is one big s-expression which contains all the glsl functions and variables.

#Layout

each element looks something like this

(((("EmitStreamVertex" "void") (("stream" "int"))))
  (("EmitStreamVertex" NIL NIL NIL NIL NIL NIL :400 :410 :420 :430
  :440 :450)))

Each item has two elements:

• A list of function definitions
• A list of information on which version are supported

Function definition

Each function definitions is laid out as follows:

• The first element is a pair of the function name as a string and the return type as a string.
• The rest of the list are pairs of the argument name as a string and the argument type as a string

Version Information

Each element in the version info list is laid out as follows:

• The first element is the name of the function and in some cases enough arg info to differentiate it from other incarnations of the same function. This is one string which sucks, so I need to parse this so we can apply the version info directly.
• The rest of the elements are the versions supported. It is either nil or a keyword specifying the version. The nils are becuase the orginal tables were rows of ticks of dashes to indicate versions supported. I turned the ticks into the version keyword and the dashes into nil. Really I need to remove the nils and I will do that very soon.

GLSL Variables

In the event that the function definition list is nil then the entry is a variable. You will find it’s name as the first element in the version table along with the versions that support it.

Right I hope this helps. This will be in a state of flux for a while as I also want to extract all documentation for each entry in glsl and I’m sure there will be tweaks to be made.

I Hope it helps someone else.

Last week was a good week for making videos, and cepl in general. With the big merge, primitive osx support and sampler objects it has been feeling like cepl is progressing very quickly all of a sudden.

I have had to throw away a couple of videos I was going to release however, as they are now very out of date. I had hoped to get full blend-mode support in last night but as I failed at that it will likely be merged in in the next 3 days. The next videos in the pipeline will be on sampler-objects and blend-modes, I will the re-record my “Intro to CEPL for people who know GL”, as this was one of the casualties of the last week’s progress.

Thanks to anyone who is reading this, Ciao

Textures in opengl have parameters that affect how they are sampled. They dictate what happens when they are magnified or minified, what happens if you sample outside the texture etc.

Sampler objects allow you to override the sampling params in a texture. One sampler object can be used on multiple textures in a single draw call.

As of last night these are now available in cepl if (>= (version-float *gl-context*) 3.3)).

You can use them like this

(defvar sampler (make-sampler :wrap #(:repeat :clamp-to-edge :repeat)))

(with-sampling ((tex sampler))
  (map-g #'some-pipeline some-stream :tx tex))

With sampling overrides a texture’s sampling params with the sampler object for the duration of the scope. The texture then reverts to it’s own sampling parameters.

The functions that are used on samplers to change their parameters also work on textures so:

(setf (lod-bias some-texture) 0.5)
(setf (lod-bias some-sampler) 0.5)

are both valid.

Next up are blend-modes. I have had to hold off making new videos for a few days as they are going out of date so fast as we keep getting new features.

IT FINALLY FUCKING WORKS!

Video coming saturday explaining in detail how to get set up on osx but the tldr is:

• run osx-sbcl-launch.sh from the terminal
• slime-connect
• (in-package :cepl)

Back soon!