Been a crazy week of headbuttng a macbookpro until a cocoa application worked followed by a fantastic demoparty (thanks everyone at kindergarten for a phenominal weekend!).

I’m finally back in lisp code (phew!) and I’m around 2/3rds of the way through a rewrite of the Varjo lisp->glsl compiler.

The reason for the rewrite now is that someone asked for a tour through the code explaining how it worked…after 5 minutes of reading I decided it could only be dark-magic holding it together so I have tried to rewrite it to take into account a few of the things I’ve learnt.

Given that I wrote it when I really didnt know anything about glsl I had a lot to change! I have also taken a few of the additions from cepl I had been hacking on and made them core features of the compiler. The big one in this category is you can define shader functions that are only included in the glsl source if they are used. This means you can make a nice ‘standard library’ of functions for your shaders and know they won’t be included unless they are used in a given shader.

There are also macros (regular for now much compiler soon) and generic functions.

I’m hopefully finishing the last of the argument validating this evening and then I think the last big things are reimplementing the special functions and rewriting the string generation. Hopefully I will be done in a week or two.

Right, enough procrastinating, back to work!

Hey folks, Another cool week or so as cepl appeared in the lisp subreddit and the videos got 980 views in one day, which for me was pretty exciting!

One of the things that came up very quickly was some issues on osx which I originally thought was down to immutable textures. So I added mutable texture storage and the problem still persisted.

Thanks to the fantastic people reporting bugs and in the irc channels I was informed that on some systems when you create a opengl context it picks a lower profile unless you specifically request a higher one. These request functions are only available in sdl2 so I have very hastily ported to sdl2.

This is obviously still a project in the alpha stage of development (is there an earlier term as I’m still working out what this project will be?) so there are plenty of bugs. If you do find them and have a chance please email me or make a bug report in github. This is a part time project but I want to get it in a state suitable for real experimentation with graphics so I will get to the bugs, just slowly sometimes :D

Righto, thanks for stopping by.

Oh almost forgot I have added a very crude compile chaining system so now when you compile a shader stage or sfun it will compile any stage or pipeline that uses it. Woo! I have doen this is a really hacky way but it proves the behaviour is nice so I will build from there.

One of the recent changes to CEPL has been to add the defsfun macro. This allows defining a shader function that doesnt belong to particular stage, if you use it in a shader it justs gets added to the code.

Here is an example of one:

(defsfun sphere ((p :vec3) (r :float)) 
  (return (- (length (* rot p)) r)))

Now the goal of Varjo was to make shader code look and feel like lisp so the following thought should have been obvious to me from the start but it wasnt. The thought is that if this is so nearly lisp then I could implement lisp versions of a few functions and have a lisp version of a shader that I can run locally for easy testing.

Again, this should have been obvious but I’m quite excited about the posibilities. As I have alreay wrapped the datatypes I can just pass the local version a c-array rather than a gpu array and get it to work.

Obviously the local versions would be too slow to use for rendering but would be great for testing and development in the repl. Also as the shader functions require types w can translate these into declares that will help with the efficiency.

This is a fair way down my list of things to do but after I have nailed time and space abstractions down a bit I will revisit this.

It has been a crazy few months. I got offered, and accepted, a job at a startup in Norway so about a month back I moved and set up in Oslo.

I’d finally settled in and suddenly CEPL turned up on reddit and people were checking it out, which was great…except I had been making loads of breaking changes in master so it was a bloody mess!

Things are really coming along well though so I need to make some videos showing how the memory handling works. I think I’ll do that tomorrow.

The most recent change I have been making is a rewrite of CEPL’s front-end to the Varjo shader compiler. Previously, when you defined a shader pipeline in CEPL you have to write the shaders inside the defpipeline form. Now they can be defined as separate entities and then the defpipeline stitches them together. Also you can define individual functions outside of the shader stages and they will get mixed in only if they are used in a stage. Along with shader macros I now have some nice tools to experiment with different ways of extending glsl.

I am going to rewrite my ray marcher example to use these features to see what effect it has on the ease of writing and reasoning about the shader pipeline.

Here is a video of the compiler stuff:

This is a photo dump from a build I just completed for a friend. I will be back to add detail to this soon. Ciao!

EQL!

EQL does look like the absolute answer to my app making prayers. It is a set of ECL bindings for QT which should mean I’m able to both use the power of real-time development, the excellent GUI designing tools of QT and still have small executables thanks to ECL.

The only issue is I can’t install it :b !

Yeah I’m having problems compiling it just now, though I have written to the author to see if he can give any advice. Whatever I find out I will post here.

It is no understatement to say I’m very excited! To be favourable on mobile devices an app really should be as small in size and memory footprint as possible, and this might just let me do both in one fell swoop!

Here’s hoping!

Right now I’ve got Ubuntu on this Nexus, I’m going to want to code something for it. It’s already set up for C and python dev but not a drop of Lisp in sight (gasp)!

In this post I’m going to get two versions up and running on the Nexus. ECL and Clozure.

To make this easier for yourself make sure you have installed ssh so you can do all of this from a real keyboard!

sudo apt-get install ssh

ECL

ECL is a great fit for this platform as it compiles down to C code so the executables are small and the tight integration with the FFI will make this very useful. Also it is super easy to install as it is in the repos.

sudo apt-get install ecl

and your done! Obviously your going to want to have more than just the repl so further down in this post we will look at installing Emacs and Slime.

Clozure

Next is Clozure. Now I’m usually working in SBCL when I’m on a regular machine (mainly as that’s what my mate set me up with when I started learning!) but Clozure (I believe) has had an Arm port for a while longer and also SBCL’s install process for Arm was a bit hairy for a beginner like me.

Setting up Clozure involves a little bit more work than ECL, but not by much.

Right so once you are ssh’d into yout tablet, you need to pull the latest version of Clozure from the repository.

   svn co http://svn.clozure.com/publicsvn/openmcl/trunk/linuxarm/ccl

   Note: I'm using the build from trunk which is not guarenteed
   stable, it may be worth trying the release version instead 
   but as I have not tested this yet I can't show the steps here.
   If you do try it, please let me know how it went!

Once that has finished we need to make a quick change to the config as, as default, Clozure for Arm7 is set up for soft floating point. We need hard. So:

 cd ccl/lisp-kernel/linuxarm/
 make clean

Now you need to open up the ‘float_abi.mk’ file in your favourite editor and uncomment the hard float line and comment out the soft float one. After the change the file should look like this:

  # This file should define FLOAT_ABI as one of "softfp" or "hard".
# If you change this, do 'make clean' to remove any object files
# compiled for the other ABI.
#FLOAT_ABI = softfp
FLOAT_ABI = hard

We are almost done! now run the following: make cd ../.. armcl –not-init

At this point you should get the clozure repl!

Now run:

(rebuild-ccl :full t)

And clozure will be rebuilt. This will take a few minutes so keep yourself busy with a little victory dance!

Editor

Wonderful, so now I want an editor to play around with. Unfortunately I am not versed in Vim so I can’t provide and info for you Vim fans but I do use Emacs so here goes!:

Well the first order of buisness is to get Emacs. Now it may seem overkill to have this on your device, you may prefer just to install swank and use your desktop but for me I’d like to ssh in an use Emacs from the command line.

 sudo apt-get install Emacs24 emacs-goodies-el slime

Now we just need to add the following to the .emacs file:

(eval-after-load "slime"
  '(progn
 (setq slime-lisp-implementations
       '((ccl ("/home/ubuntu/.ccl/armcl"))
	 (ecl ("/usr/bin/ecl"))))
 (slime-setup '(
		slime-asdf
		slime-editing-commands
		slime-fancy-inspector
		slime-fontifying-fu
		slime-fuzzy
		slime-indentation
		slime-mdot-fu
		slime-package-fu
		slime-references
		slime-repl
		slime-sbcl-exts
		slime-scratch
		slime-xref-browser
		))
 (slime-autodoc-mode)
 (setq slime-complete-symbol*-fancy t)
 (setq slime-complete-symbol-function
  'slime-fuzzy-complete-symbol)))

(require 'slime)

Done!

Well thats all done! I’ll try and keep this updated with any progress I make on the tablet. I’m very excited to see what happens next. Ciao

Given the above blurry video of Ubuntu on the Nexus 7 and knowing that in it’s current state, it will be bug ridden, unoptimised and dangerous to ones sanity, why would someone go shell out £170 for one?

Well I just have so let me explain my logic! I use android, I like it, I have no hatred of Apple or Google but while I have moved more of my computing life into software that is open, I have moved more of my personal life into services which are not.

This has been growing on my mind more and more lately and while I’m not ready to completely pull myself from these services I do know that I must start looking. There have been plenty of news stories recently which should serve as warning to where these services are heading and the kind of problems which will increasingly arise.

My phone is my biggest and most continuous link to proprietary services, namely google mail, search, maps, etc. By following along with the development of Ubuntu on mobile devices I hope I can lay the groundwork for my own move to an open source based mobile life.

Again I don’t believe that Google are using my data for malevolent purposes…But I do believe that as the market place changes companies try and leverage what they have in new ways to make money. I don’t want to be that future administration’s leverage.

Another, very important reason is that it will be really cool! I want to see Ubuntu become a viable alternative to what exists already, and look forward to all advances and experience that will move all around the Linux ecosystem from having a solid Debian based OS being transformed into a solid mobile platform.

Bring it on!

p.s. I also want to port CEPL to OpenGl ES and get lisping on this thing!

I have just started looking for resources on writing domain specific languages in common lisp and stumbled upon this excellent video by Rainer Joswig on the subject. He gives a quick run through and his explanations (as always with Rainer) are clear and illuminating.

I’m also buzzing at the idea of eventually getting my brain into a place where I move as naturally through a problem as Rainer shows in this video. All to often I feel the apprehension caused by not remembering just to let go and just start writing…I guess this affects all disciplines in some form or another but I am beginning to appreciate how differently I can code in Lisp to how I did in Python for example.

How much is down to the language and how much is just me developing as a programmer remains to be seen but it certainly feels different.

Enjoy the video.

BWUHHAHA I’m back! I have spent the last while working on code that really doesnt make for good videos so I have been quiet but I have something new to show.

This video shows a really simple directional light acting on the monkey head model.

What I’m really pleased with is that I didn’t need to be rigorous which checking that the code would work before I ran it. I was able to run it an mess around with things until they worked. A good example was with the ordering of the matrix multiplications. If you have done some of this stuff yourself you’ll know that, unlike multiplying regular numbers the order you multiply matrices is very important to the outcome so while:

 2 x 4 = 4 x 2 

..It is not the same when using matrices. I frankly was confused with the orders when I first ran the demo and it showed! The monkey head was all over the place and the light wasn’t working. What was wonderful was I was able just to keep swapping things around until I got it right. I caused a lot of bugs in the process but, as you are still able to compile parts of your code while the debugger is waiting, I was able to undo my mess and carry on running the demo as if I wasn’t incompetent!

All in all I’m pretty chuffed with all of this and have plenty more to show too. I will defintely have to write a long post about all the work I have been doing on the opengl wrapper part of CEPL, it makes the code much easier to write, and read for that matter!

Well that’s all for another day,

Take care.