TAKASHI'S WORKSPACE

Various examples in Haskell's FRP.Reactive

2011-12-15T11:17:00.000-08:00

After playing with Flapjax library in Javascript, I moved to Reactive to learn more about FRP. Because research on Functional Reactive Programming is most active in Haskell, I thought it would be better to do that. Reactive seems to be a nice library, but unfortunately I couldn't find many working code examples. So I show some of them as my exercise. To write this, I owe a maoe's great article in Japanese.

As I didn't have much time, I couldn't write a good explanation now. But still I hope it helps some people who learn Reactive like me. I used Haskell Platform 2010 (slightly old) and did cabal install reactive --enable-documentation to install Reactive.

The first example shows "Hello, World!" after three seconds. atTime generates a timer event, and <$> convert this event to IO action (\_ -> putStrLn "Hello, World!") which writes a string.

This is as same as above, but it makes events each second.

This makes running Fibonnaci numbers. You can use scanlE to process previous value and current value of the event in a function. In this case, (0, 1) is the initial value, and when an event occurs, the function \(n0, n1) _ -> (n1, n0 + n1) calculates next value, and the result (the first case is (1, 1)) is used as a next argument when a new event occurs.

It shows characters as you type. It looks difficult but you don't have to worry about run function. The important part is machine :: Event Char -> Event (IO ()) that convert a character input event to an IO action.

This example shows how to merge two events. onType is same as machine in the previous example, and onClock is same as helloMany.hs example. I used `mappend` to merge the two events

This shows a simple state machine. The function next defines the state machine, and mealy_ convert the definition to an event. zipE is another way to merge two events. Unlike mappend, you can see two values in the two events in a same time.

Flapjax vs Tangle

2011-12-06T01:00:00.000-08:00

Functional Reactive Programming (FRP) is a framework to deal with time-varying data in a clean way. It is a combination of beauty of functional programming and dynamics of object oriented programming. The basic principle is easy enough as spreadsheets, however, its vague scope and arcane terminologies keep you from grasping it. It's not quite easy to answer the question such as what makes FRP different from Observer Pattern, Data Flow, etc ??. I think a good way to explain FRP is to compare FRP library against non-FRP library, and I could show you where FRP is special, and pros-and-cons of FRP.

I examined Flapjax as an example of FRP, and took Bred Victor's Tangle as the comparison target. Although Tangle has similar goal of FRP as he wrote "Tangle is a library for creating reactive documents", its implementation is quite different from Flapjax.

Flapjax: Side-effect is hidden inside the framework. Time-varying data is represented by dependent tree, and you can compose those trees to implement a complex behavior.
Tangle: Tangle provides a simple framework and UI widgets, but the data flow is represented by a normal imperative programming and assignments.

Because of those properties, I think comparing the two libraries is helpful to understand what FRP is. I hope it makes clear idea about FRP in your mind.

Simple Calorie Calculator in Tangle

This is the first example from the Tangle's documentation. You can modify the number of cookies by dragging, and it keeps calculating the calories as you change the value.

When you eat cookies, you will consume calories.

To make this nice reactive document. This document consists with two parts, HTML for the view and javascript for the model.

<p id="tangle"
  When you eat <span data-var="cookies" class="TKAdjustableNumber" data-min="2" data-max="100"> cookies</span>,
  you will consume <span data-var="calories"></span> calories.
</p>

The HTML part is straightforward, this is just a normal HTML except special attributes for Tangle. Data-var is used to connect HTML elements to Tangle object's properties. Class name TKAdjustableNumber makes a draggable input control. Data-min and data-max are its parameters.

var element = document.getElementById("tangle");

new Tangle(element, {
  initialize: function () {
    this.cookies = 4;
  },
  update: function () {
    this.calories = this.cookies * 50;
  }
});

The actual model of the document is described in the second argument of Tangle object's constructor (new Tangle). It consists with just two parts. initialize sets up the initial state, and update is invoked whenever you modify the input value. Tangle connects the model and the HTML element specified by getElementById("tangle").

This initialize-update structure is fairly common among end-user programming language like Processing and Arduino.

Simple Calorie Calculator in Flapjax

Let's move on to Flapjax. Unfortunately, Flapjax doesn't have a nice input widget as Tangle has. Instead, we use a traditional input field. But other than that, the behavior is identical.

When you eat cookies, you will consume calories.

As Tangle, the Flapjax version has HTML part and Javascript part. Note that Flapjax provides "Flapjax Syntax" which allows you to write a simpler notation, but we don't use it because I want to compare those as Javascript libraries.

<p id="flapjax" class="example">
  When you eat <input id="cookies" value="4" /> cookies,
  you will consume <span id="calories"></span> calories.
</p>

Flapjax's HTML part is similar as Tangle's. The element identifiers (cookies and calories) are given by id attributes. Unlike Tangle, the initial number of cookies is written in the input field.

var behavior = extractValueB("cookies");
var colories = behavior.liftB(function (n) { return n * 50; });
insertDomB(colories, "calories");

In Flapjax, time-varying data is called behavior. The goal of the program is to make a behavior which always calculates calories of the cookies. It's not so difficult than it seems. ExtractValueB creates a behavior from a form element, in this case, extractvalueB("cookies") tracks every changes happening in the input field named "cookies". This created behavior is processed by the function at the argument of liftB, in this case, whenever you modify "cookies" field, colories represents a value which is always 50 times by the number of cookies.

Eventually, insertDomB insert the content of colories where HTML element "calories" is and the calories are shown on the screen. This element is automatically updated.

Unlike Tangle, there is no side-effect in the program. One advantage of FRP is that you are not confused between old values and new values. In Tangle's example, this.cookies is old value (input) and this.calories is new value (output). But you are free to be mixed up those. In Flapjax, a new value is always the return value of a function, and there is no chance to be mistaken.

Implement Adjustable Number Widget in Flapjax

One of advantages of FRP is its composability. You can make a complicated behavior by combining simple behaviors (occasionally, imperative programming gives you a hard time for debugging if the bug involves with connected program modules with side-effects). To demonstrate this feature, I will show you how to make a Tangle-style draggable widget in Flapjax. This problem is particularly interesting because processing drag and drop involves a state machine, but a state machine is not quite fit with a functional programming style. So you might find pros and cons of FRP clearly from this example.

When you eat cookies, you will consume calories.

The HTML part is almost identical except adjustable class in the input field which points a Tangle like (but not fashionable enough) stylesheet.

<p id="flapjax-drag" class="example">
  When you eat <input id="cookies-drag" value="4" class="adjustable"/> cookies,
  you will consume <span id="calories-drag"></span> calories.
</p>

The main Javascript part is also similar as above. But in this time, we are implementing makeAdjustableNumber to make a draggable widget from the element named "cookies-drag".

var element = document.getElementById("cookies-drag");
var behavior = makeAdjustableNumber(element);
var colories = behavior.liftB(function (n) { return n * 50; });
insertDomB(colories, "calories-drag");

A drag gesture consists of three events, mousedown, mousemove, and mouseup. After a mousedown is detected, it has to track mousemove events to know how far you are dragging. You can make such a state machine to construct a higher order event stream. Here are two new concepts. An event stream is similar as behavior, but it is a stream of discrete events instead of continuous values. But you don't have to worry about that. It's just another object which has slightly different API. A higher order event stream is an event stream of event streams. This is used to make a stream which behavior is switched depends on the input.

This mouseDownMove makes a higher order event stream that tracks mousedown and mousemove. extractEventE(element,"mousedown") extracts mousedown event in the element. When the event signaled, the function inside the mapE is evaluated. MapE is similar as liftB but it is only for an event stream. Inside the function, extractEventE(document,"mousemove") find mousemove events and track the distance from mousedown. Note that I used document to find the event because occasionally you drag a mouse to outside the widget.

function mouseDownMove (element) {
  return extractEventE(element,"mousedown").mapE(function(md) {
    var initValue = parseInt(element.value);
    var offset = md.layerX;

    return extractEventE(document,"mousemove").mapE(function(mm) {
      var delta = mm.layerX - offset;
      return Math.max(1, Math.round(delta / 20 + initValue));
    });
  });
}

We need to handle mouseup event also. The mouseUp function returns a higher order event stream that find mouseUp event and the zeroE happily does nothing.

function mouseUp (element) {
  return extractEventE(document,"mouseup").mapE(function() {
    return zeroE();
  });
}

And these two event stream make by mouseDownMove and mouseUp are going to be merged by the mouseDownMoveUp function to complete a mousedown, mousemove, and mouseup cycle. MergeE is used to merge two events streams. We need one more step switchE to convert a higher order stream to a nomal stream, in this case, a stream of numbers (distance).

function mouseDownMoveUp(element) {
  var downMoveUp = mouseDownMove(element).mergeE(mouseUp(element));
  return downMoveUp.switchE();
}

Finally, we connect the event stream into an HTML element. Here I did slightly dirty work. Whenever a drag gesture happens, the element.value attribute is set. Probably using insertDomB to make an output element is cleaner way, but I chose this dirty way to make it simple. At the last line, the event stream is converted to a behavior object by startsWith. And that's how makeAdjustableNumber is implemented.

function makeAdjustableNumber (element) {
  var drag = mouseDownMoveUp(element);
  drag.mapE(function(n) { element.value = n; });
  return drag.startsWith(element.value);
}

Honestly, Flapjax doesn't seems to be too easy to use. But part of the reasons might be that I chose to show a plain Javascript syntax to introduce the mechanism. Flapjax also provides its own compiler which provides cleaner syntax. This Flapjax syntax should improve readability a lot. Anyway, I hope this short note helps you to grab a brief idea of Flapjax and FRP.

References

Yet Another "Alligator Eggs!" Animation

2011-09-22T19:49:00.000-07:00

Bret Victor came to our office yesterday, and we had a great chat. He is a great thinker and has a beautiful sense about visualizing abstract ideas. I really like his works. I want to learn his idea more, but as a starter, I tried to implement his early famous Alligator Eggs! game. This game was made to teach about lambda calculus to eight years old kids. But it's even more fun to adult hackers!

Alligator and an egg : λx.x

This is a green alligator and her egg. This family shows a lambda expression λx.x (because I know you are not an eight years old, I use formulas without hesitation!). There is a no animation as there is nothing to eat.

An alligator eats an egg : (λx.x) y

But things are getting fun when there is something to eat before the alligator mother. In this case, a blue egg. If you click on the diagram, you see what's happening (I only tested Chrome, Safari, and Firefox). The alligator eats the poor blue egg. But price for the sacrifice is too high. The mother will die, and we will see the new baby.

And then, things are getting curiouser. The new baby doesn't look like the mother at all, rather it is like a blue egg, the victim of the slaughter. What's a amazing nature of the lambda land!

Take first : (λx.λy. x) a b

This is slightly a hard example. There are two alligators "x" and "y", and two victim eggs "a" and "b" on the right side. If there are more than two things next to an alligator, the alligator eats left one first (it is called as left associative in jargon). Can you guess what does happen after the meal? Alligator "x" eats egg "a", and alligator "y" eats egg "b". And only egg "a" survives (because it transmigrates through the green "x" egg).

You can think that this alligator family (λx.λy. x) eats two things and leave the first one. In a same way, can you think of an alligator family which eats two things and leave the second one? Here is the answer.

Old alligator : (λx.x) ((λy.y) (λz.z))

There are a few things to know more. Old alligators are not hungry. But they keep guarding their family while they guard more than one things. They behave like parenthesis in a lambda expression.

Color rule : (λx.λy.x) (λy.y)

This rule is the most tricky one. There are two blue alligators "y" at left and right, but those two are not in a same family. The only mother of the blue egg "y" is the right one. It gets trickier when the family is eaten by the green alligator because the blue family is reborn at the green egg is, where is bottom of another blue alligator. To make them different, the right blue family change the name and color to "y1" and orange.

Omega (Mockingbird hears the Mockingbird song) : (λx.x x) (λx.x x)

By these rules, you can make various kinds of alligator ecosystem. This is my favorite one. (λx.x x) is called a "Mockingbird" or, rather we should call it Mockingalligator. It doubles its prey twice. So what happens if a mockingalligator eats a mockingalligator? The result is called one of omegas, an infinite loop. They are eating forever. To stop the endless violence, please click the diagram again. But please do not to click three times! Because of my bug, something wrong will be happening.

Y combinator : λg.(λx.g (x x)) (λx.g (x x))

This is dangerous but beautiful one. The omega ecosystem above kills each other but it doesn't make any, but this Y combinator is very fertile. It produce many, so you have to watch it carefully, otherwise it consumes all the CPU power you have eventually!!

3 + 4 : (λa.λb.λs.λz.(a s (b s z))) (λs.λz.(s (s (s z)))) (λs.λz.(s (s (s (s z)))))

Actually, alligators also can do serious jobs. If you design carefully, you can teach them how to calculate 3 + 4! In this example, the middle family represents three and the right family represents four (count green eggs). And the result is a family with seven green eggs! This is called Church numbers (I don't have a time to explain the theory, so please read the link).

I only introduced very few alligator families. If you want play it, visit http://metatoys.org/alligator/ and design by your self. You can also download from http://github.com/propella/AlligatorEggs. The source code is messy because I haven't written javascript recently, but I'll clean it up soon.

A hidden story behind the EToys Castle

2011-07-09T19:21:00.000-07:00

http://metatoys.org/demonCastle/

If you have played with Etoys, you might have seen The Etoys Castle (or The Demon Castle) tutorial. But you would never know how the story ends, because the Etoys distribution only includes the first chapter, and the last slide shows "To Be Continued ...". However, there are actually the hidden sequels, and the story has a happy ending.

When I first wrote the story in 2006, there were three chapters. The first chapter was about learning "handles", the second one was about the painter, and the third one was about scripting. But due to some technical issues, I gave up to publish them. Today, I happened to clean up my hard drive and I found old files. It's shame that I have never published rest of them. So I gathered the screen shots and made up one page html.

Tamacola (5)

2010-09-24T00:45:00.000-07:00

Tamacola is not just another LISP language, it is designed as a meta-language to make a new language. I'll explain this feature today. Today's goal is to design a subset of lisp language. If you think that a lisp is too simple to keep your passion, sorry, be patient, simple thing first.

Prepare your Tamacola environment

To setup Tamacola environment, you need to download both Tamacola distribution and Tamarin VM. Those are available on http://www.vpri.org/vp_wiki/index.php/Tamacola. You need add the PATH environment variable to find the avmshell command, and also it would be useful to set the PATH to bin/ in the tamacola tree. To make sure Tamacola works, plese type:

make run-example

It runs all of the examples in the Tamacola distribution as well as recompile the compiler. If you don't find any error, you are ready to go. Otherwise, please let me know the problem.

Tamacola command

Tamacola command read a tamacola program and run immediately. If you want to make a Flash contents, another command tamacc (Tamacola Compiler) is more suitable. Now we are playing with an interactive shell of tamacola command, so I'll give you a brief explanation. The interactive shell starts with minus (-) option. Let's try a simple arithmetic. If you didn't setup PATH environment, please specify the directory name, too.

$ tamacola -
Cola/Tamarin
> (+ 3 4)
7

You can also give Tamacola source files as well as compiled binary names. Typically, source code ends with .k, and a binary ends with .abc. Tamacola is smart enough to detect newer file between .k and .abc.

Match against a string constant

Suppose you are on some working directory, and you have already set PATH environment to the bin/ directory. And then, we are going to write a very simple language, greeting:

;; greeting.g - A simple PEG example

greeting = "morning" -> "Good Morning!"
         | "evening" -> "Good Evening!"

This stupid example answers "Good Morning!" if you say "morning", and it answers "Good Evening!" if you say "evening". This PEG syntax is easy to understand. The right hand side of = is a rule name. A rule name is translated as a function once it is built. -> means an action rule, where if the left hand is matched the right hand side is returned. | is an Ordered options. In this case, the parser tries the first case "morning", and tries the second case "evening" only if the first case fails.

Save this syntax named "greeting.g". To test this language, type those commands:

$ mkpeg greeting.g
$ tamacola greeting.k -
compiling:  greeting.k
Cola/Tamarin
> (parse-collection greeting "morning")
"Good Morning!"
> (parse-collection greeting "evening tokyo")
"Good Evening!"

Mkpeg command converts grammar file (greeting.g) to tamacola source (greeting.k), a rule "greeting" the result can be read by tamacola shell. Greeting.k is built on the fly and the command prompt is shown.

Parse-collection's first argument is a parser name (in this case "greeting"), and the second is a input collection. As the name implies, it accepts any collection as the input stream.

The second case shows an interesting property of PEG syntax. Although the second rule matches the beginning part of the input "evening tokyo", still the input remains more string " tokyo". PEG doesn't care if the input is competely consumed or not. If you really want to make sure that the entire input is matched, you need to explicitly tell the Parser the point where end of the file.

Number parser

The last example only matched a predefined constant, but we make a parser for any integer number here.

;; number.g -- A number parser

digit   = [0123456789]
number  = digit+

We also convert the grammar specification into the tamacola program, but in this case, we give -n option to tell the namespace. A namespace is useful when you want to use a common name as a rule name like "number". Because "number" is already used in the system, you can not use it without namespace.

The grammar itself is easy to understand if you have an experience with regular expressoins. Brackets ([]) matches one of characters inside, and postfixed plus (+) repeats previous expression with one-or-many times.

$ mkpeg -n number number.g 
$ tamacola number.k -
compiling:  number.k
Cola/Tamarin
> (parse-collection number/number "xyz")
FAIL
> (parse-collection number/number "345")
{token-group:
(53 52 51)}

Because we use the namespace "number", we need specify the namespace before slash(/) in the function name.

As you might notice, this parser correctly rejects a non-number like "xyz", and accepts "345". But the result is not so useful. The return value of plus is a special object named "token-group", but we would want a number represented by the string, instead. So we put a conversion function to get the value.

number  = digit+:n      -> (string->number (->string n))

$ tamacola number.k -
compiling:  number.k
Cola/Tamarin
> (parse-collection number/number "345")
345

Now parser returns a number conveniently. Perhaps you might think that it is somewhat cheating. As the string->number function itself is a kind of number parser, we should have write a number parser without string->number! Yes we could. But it leads more interesting topic about left and right recursion, so I leave it for later.

S-expression parser

Now we are going to write a parser for almost real S-expression. This parser can only handle number and list, but it is useful enough to explain the essence of Tamacola.

;; sexp.g
;; Lexical Parser

spaces  = [ \t\r\n]*

digit   = [0123456789]
number  = digit+ :n spaces              -> (string->number (->string n))

char    = [+-*/abcdefghijklmnopqrstuvwxyz]
symbol  = char+ :s spaces               -> (intern (->string s))
        
sexp    = symbol
        | number
        | "(" sexp*:e ")"               -> (->list e)

In this grammar, only new operator is the postfix star (*) which repeats zero-or-many times. Rest is straightforward. To test this grammar, we use Tamacola's simple test framework. Writing test case is better than the interactive shell, because you don't have to type same expression many times.

;; sexp-test.k

(check (parse-collection sexp/spaces "    ")            => 'SPACES)
(check (parse-collection sexp/digit "0")                => 48)
(check (parse-collection sexp/number "345")             => 345)
(check (parse-collection sexp/char "a")                 => 97)
(check (parse-collection sexp/symbol "hello")           => 'hello)

(check (parse-collection sexp/sexp "345")               => 345)
(check (parse-collection sexp/sexp "hello")             => 'hello)
(check (parse-collection sexp/sexp "(hello world)")     => '(hello world))
(check (parse-collection sexp/sexp "(3 4)")             => '(3 4))
(check (parse-collection sexp/sexp "(print 4)")         => '(print 4))

The check function comes from SRFI-78. This function complains only if the left hand value and the right hand value differ. Otherwise, does nothing. I like this UNIX stile conciseness.

As a convention, a test program is added a postfix "-test" with the main program's name. I borrowed this custom from Go language.

Make sure this program do nothing.

$ tamacola sexp.k sexp-test.k

Lisp Compiler

The PEG parser can handle any list structure as well as string. It allows you to write compiler in PEG. In a string parser, the input is a string and the output is some object (a list in our case), but in a compiler, the input is a lisp program and the output is a assembler code.

;; Compiler

arity   = .*:x                          -> (length (->list x))
insts   = inst* :xs                     -> (concatenate (->list xs)) 
                                        
inst    = is-number:x                   -> `((pushint ,x))
        | is-symbol:x                   -> `((getlex ((ns "") ,(symbol->string x))))
        | '( '+ inst:x inst:y )         -> `(,@x ,@y (add))
        | '( '- inst:x inst:y )         -> `(,@x ,@y (subtract))
        | '( '* inst:x inst:y )         -> `(,@x ,@y (multiply))
        | '( '/ inst:x inst:y )         -> `(,@x ,@y (divide))
        | '( inst:f &arity:n insts:a )  -> `(,@f (pushnull) ,@a (call ,n))

There are some new elements in the grammar. Quoted list '( ) matches a list structure, and a quoted symbol matches a symbol.

A prefix ampersand (&) prevents to consume the stream even if the rule matches. For example, &arity rule examine the rest of the list, but the contents are matched again by the insts rule later.

Is-number is matched against number, and is-symbol is for a symbol. Those rule can not be described as PEG grammar, but as a lisp function.

(define is-number
  (lambda (*stream* *parser*)
    (if (number? (peek *stream*))
        (begin (set-parser-result *parser* (next *stream*))
               #t)
        #f)))

(define is-symbol
  (lambda (*stream* *parser*)
    (if (symbol? (peek *stream*))
        (begin (set-parser-result *parser* (next *stream*))
               #t)
        #f)))

A rule is a function which receives the stream and the parser (an object which store the result). The rule function returns #t if it matches, and #f if it fails.

I think it is easier to see the test code than read my explanation.


(check (parse-collection sexp/arity '(a b c))   => 3)

(check (parse-collection sexp/insts '(3 4)      => '((pushint 3)
                                                     (pushint 4)))

(check (parse-collection sexp/inst '(3))        => '((pushint 3)))

(check (parse-collection sexp/inst '((+ 3 4)))  => '((pushint 3)
                                                     (pushint 4)
                                                     (add)))

(check (parse-collection sexp/inst '((f 3 4)))  => '((getlex ((ns "") "f"))
                                                     (pushnull)
                                                     (pushint 3)
                                                     (pushint 4)
                                                     (call 2)))

Put it in an envelope

We still need a little bit to construct a real assembler code. This detail topic is out of the context, so I simply show the code.

program = inst:x  -> `(asm
                       (method (((signature
                                  ((return_type *) (param_type ()) (name "program")
                                   (flags 0) (options ()) (param_names ())))
                                 (code ((getlocal 0)
                                        (pushscope)
                                        ,@x
                                       (returnvalue))))))
                      (script (((init (method 0)) (trait ())))))

And the test case.

(check (parse-collection sexp/program '((print 42)))
       => '(asm
            (method
             (((signature ((return_type *) (param_type ()) (name "program")
                           (flags 0) (options ()) (param_names ())))
               (code ((getlocal 0)
                      (pushscope)
                      (getlex ((ns "") "print"))
                      (pushnull)
                      (pushint 42)
                      (call 1)
                      (returnvalue))))))
            (script (((init (method 0)) (trait ()))))))

You can read the entire program in example/sexp.g in the Tamacola distribution. To try the program, please enter:

make -C example test-sexp

Left recursion

We left an interesting topic about left and right recursion. Let me show you our number parser again.

digit   = [0123456789]
number  = digit+:n               -> (string->number (->string n))

If we don't want to use string->number function, I would write the parser as:

;; Use fold-left
digit1   = [0123456789]:d        -> (- d 48)
number1  = digit1:x digit1*:xs   -> (fold-left
                                      (lambda (n d) (+ (* n 10) d))
                                      x
                                      (->list xs))

Digit1 rule converts the ascii value of the the digit character, and number1 rule construct a decimal number. As you see, you need to use fold-left function to construct a number because a number notation is essentially left recursion. For example, a number 34567 actually means:

(((3 * 10 + 4) * 10 + 5) * 10 + 6) * 10 + 7

However, PEG parser doesn't parse left recursion grammar in general. So I had to reconstruct the left recursion structure by fold-left. This is not hard at all if you familiar with functional programming. In functional programming, a list is considered as a right recursive data structure and it is even natural that a list is parsed by a right recursive way. However, I admit that it looks awkward for some people.

Yoshiki Ohshima provides a very useful extension to support a direct left recursion. To use his extension, the number parser is written as:

;; Use left-recursion

digit2   = [0123456789]:d        -> (- d 48)
number2  = number2:n digit2:d    -> (+ (* n 10) d)
         | digit2
number2s = number2

You need to load runtime/peg-memo-macro.k to use this extension.

$ tamacola ../runtime/peg-memo-macro.k number.k -
Cola/Tamarin
> (parse-collection number/number2s "345")
345

The real parser and compiler are bigger than presented grammars here, but I explained all of the essential ideas. I hope it helps you to make your own language!

Tamacola (4)

2010-09-15T22:47:00.001-07:00

Tamacola in Tamacola

After I made the Tamacola compiler written in COLA, next thing to do was to implement it in Tamacola itself. A language is called self-hosting if the language is written in the language itself. This implies various advantage.

First, once self-hosting is done, you don't need to use COLA anymore, you can improve or modify any language aspects on Tamarin VM. If I carefully design the environment, it would be possible to do language design only on the Web browser (it needs server side help for security reason, so it hasn't done yet).

Second, self hosting is a benchmark for the language to tell that it is good enough. Scheme is especially simple language, so there are a lot of people who implement toy-Scheme. But because my Tamacola is now self-hosting, I could proudly claim that this is not a toy! Well, this is rather self satisfaction, though.

Third, it provides a rich library including "eval" function. A compiler uses various programming techniques, and those must be useful for other programs, too.

To make it self-hosting, there were two key problem which are macros and eval.

Bootstrapping Macros

I heavily used macros in my compiler, for example, the parser written in PEG was converted a bunch of macro expressions. The problem is, expanding macros requires eval function but I wasn't able to make eval before the parser was done. It's a deadlock! Here is a typical macro written in COLA:

(define-form begin e (cons 'let (cons '() e)))

This is how the macro works. When the compiler find a expression like:

(begin
  (print "Hello")
  (print "World"))

Expressions inside begin is bound to e, the body (cons 'let (cons '() e)) is executed in compile time and the expression is expanded to:

(let ()
  (print "Hello")
  (print "World"))

Such expansion is impossible without eval function because the compiler need to evaluate a list (cons 'let (cons '() e)) given by user. What I would do when I didn't have eval yet. But I realized that macros only include basic list functions like car, cdr, and cons in many cases. And a more complicated macro could be hard corded as a special form in the compiler. So I invented a pattern base macros.

(define-pattern ((begin . e) (let () . e)))

Basically this is a subset of Scheme's syntax-rule. If the compiler finds an expression starting with begin, rest of the expression is bound to e and substituted as a right hand side. Those expansion requires only limited set of list functions, so the compiler doesn't have to provide full eval function. This macro syntax made my compiler readable, and I was able to continue happily.

Even after I implemented more dynamic traditional macro with eval function, I keep using this pattern base macros mainly.

Eval

To implement eval function, you need to understand the dynamic code loading facility provided by the VM. Note that this is not part of AVM2 specification, and Avmshell (a console Tamarin shell program) and Adobe Flash have different API.

Avmshell has straightforward API. You give compiled byte code, and the function returns the value. Because Tamacola is now written in Tamacola, you can invoke the compiler as a library function and get byte code you want to execute.

avmplus.Domain.loadBytes(byteArray:ByteArray)

You can get the domain object by Domain.currentDomain() static method. Those useful functions in Avmshell are found shell/ directory in the Tamarin-central repository.

Flash Player has somewhat tricky API for dynamic code loading. The signature is normal.

flash.display.Loader.loadBytes(bytes:ByteArray, context:LoaderContext = null):void

There are two problems for our purpose. First, this method is not designed mainly for dynamic loading, it only accepts SWF, JPG, PNG, or GIF files, and byte code happen to be accepted inside a SWF file. So I had to construct SWF file to load code. In case if you don't know about SWF file, SWF file is a kind of container format. You can embedded vector graphics, mp3 sounds, and ActionScript byte code. Making a SWF file is not particularly difficult though, it needs nasty bit fiddling.

Second, this is far more problematic, is that this method works as asynchronously. In other words, this doesn't return the result value. Instead, you need to give it a callback function to wait to finish the code. Additionally, this method doesn't return value at all, so if you want the return value, you need to setup some explicit return mechanism by yourself.

Practically, this cause a problem if you want to write a traditional macro definition and use the macro in a same source code. Because a traditional macro need to evaluate a lisp expression in a compile time, but the eval function doesn't return before the compilation thread is done. I could solve the problem by setting up compilation queue or something, but it would cost performance penalty which I don't want. And now I simply gave up.

I have explained pretty much all interesting aspect of the self hosting compiler. I'll talk about how to make a new language on the Tamacola environment later.

Tamacola (3)

2010-09-14T23:03:00.000-07:00

How a lisp program is compiled to Tamarin VM

Now I'm going to talk a bit about how a lisp (almost Scheme) program is compiled into Tamarin's byte code. This topic is especially interesting if you are curious to make your own language or VM.

Tamarin VM is made for ActionScript, so its byte code is also specifically designed for ActionScript. In other words, it is a slightly tricky to implement other language than ActionScript. In case if you don't know about ActionScript, it is almost identical as JavaScript in the execution model. Difference between them is about optimization with explicit type notion and static field.

ActionScript and Scheme are common for those aspects:

Lexical scope.
Function object.
Variable arguments with no curring.
Dynamic typing (a value has type, not variable).

But there are significant difference.

ActionScript doesn't have a simple function call. Everything is a method call.
In ActionScript, a function has a scope. No scope block or let expression.
Tail call optimization is not supported.
Call stack can not be accessed.

Those limitations sound like that Tamarin VM is inferior. But no, actually those limitations come from Tamarin VM's advantage and optimization. If you happen to have a chance to design your VM, please learn from the lesson. There ain't no such thing as a free optimization. Any optimization kills some generality. I'll explain each case.

ActionScript doesn't have a simple function call neither Tamarin VM. This is rather harmless though. When you see a function like expression like trace("hello"), this is actually mean (the global object).trace("hello"), and eventually, the receiver passes to the function as the first argument. In other words, if you want to construct a function call with two arguments, you need to make three arguments where the first argument is "this" object. A slightly tricky part is primitive operators like + or -, which don't have "this" object. Those primitives are special case.

ActionScript also has lexical scope, but only a function has a scope. So I have to be careful when I compile let expression in Scheme. Most simplest way to implement a let expression is to use a function. A let expression can be always translated to a lambda in theory though, this is a huge performance disadvantage. So I use "with" expression in ActionScript. "With" expression is an unpopular syntax in ActionScript, but you can use any object as a scope object. I borrowed this idea from Happy-ABC project http://github.com/mzp/scheme-abc.

Lack of the tail call optimization in Tamarin VM was the most disappointed thing to me. It prevents a functional programming style. I simply gave up it. Tail call optimization is not difficult topic at all. If the target were a native code like x86, it would be a matter of swapping stack and jump. But Tamarin VM doesn't allow direct access of stack or jump to other function. I understand that it might cause a security issue though, it would be wonderful if VM would provide special byte code for tail call.

Finally, you can't access the call stack directly, therefore you can't implement call/cc. The reason why I can't call Tamacola as Scheme is the lack of tail call optimization and call/cc. It prevents many experimental language features like generator, process, or so. But considering rich libraries provided by the Flash API, I would say Tamacola will be a reasonably useful language eventually.

I'll tell you convolved self hosting process and macros tomorrow.

Tamacola (2)

2010-09-14T09:43:00.001-07:00

COLA

Once the assembler was done, I was able to test verious Tamarin VM's features, even I wrote a tiny GUI application on Adobe Flash in the assembler. Then next step is the compiler.

Another goal of the project was to port Ian Piumarta's COLA framework to Tamarin (the project name came from this). And perhaps this is only the real technical contribution of the project. COLA is a meta language (a programming language to design another language) which resembles Scheme. COLA has a nice sub-language called Parser Expression Grammar that makes parser very terse. My plan was to write a boot-strappng compiler in PEG and COLA, then to implement COLA library, and to write the real compiler in PEG and Tamacola itself.

I won't give you the detail of PEG. But briefly, it is as simple as a regular expression and as powerful as a context free grammar.

When that time I started writing the compiler, COLA has no library at all except PEG framework, so I needed to write necessary libraries by myself from scratch. Fortunately COLA has quite a powerful external function call feature (a kind of FFI), macro sysytem, and a flexible object oriented framework. So writing library is not so hard. But I tried not to use COLA specific features as possible because it would be a problem when I rewrite the compiler in Tamacola itself later.

To implement the library, I borrowed function specifications from R6RS as well as possible to avoid unnecessary confusion. There were exception because COLA treat a slash "/" character as special for namespaces, I took PLT's function names in this case.

Writing lisp libraries is interesting puzzle to me because there were some requirements and constrain for the domain. Those requiments are:

Unit testing framework.
Library framework.
List manipulations.
String functions.
Bit operations and streams.
Pretty printer for debugging.

These requirements were carefully chosen. Because COLA has only modest debugging facility, the unit test framework must be there. So my first goal was to implement all functions needed by the unit testing. I needed a pretty printer for debugging, too.

Another "must have" library was bit operators, and file / in-memory streams that is needed to the assembler. Interestingly enough, R6RS doesn't define enough functions to support those. For example, there are no portable way to specify a stream to be binary or text. So I needed a bit creativity.

Eventually, I wrote all libraries and the compiler. And I got a pretty good sense about a minimun set of functions needed for compiler, which are testing framework, pretty printer, bit operators, and streams. In other words, if your language has those features, your language can be self-hosting.

The real puzzle part was the order. Those requirements must be precisely ordered by need. For example, the pretty printer must follow stream and string functions because the pretty printer uses those functions. Although you can write functions in random order as you like in Lisp, precise order makes testing and debugging is easy. I kept this discipline. I even implemented the test library twice, the first one was concise assert function, and the second one has more friendly fail message by the pretty printer.

It took a few weeks to build a simple compiler, but still there were long way up to the point where self-hosting can be done. One thing that I had learned from the stage was, even without debugger, debugging is not so hard if you have enough test cases and a good pretty printer.

Tamacola (1)

2010-09-12T16:40:00.000-07:00

Intro

I have published the source code of Tamacola, a lisp compiler which runs on Adobe Flash / Tamarin VM (or Adobe Virtual Machine 2) http://github.com/propella/tamacola. I'm pretty sure that the current version is useless if you are just looking for a lisp implementation on Tamarin (las3r and scheme-abc are much better), but Tamacola includes abundant tips if you are interested in making a self-hosting compiler on Tamarin VM. That's why I decided to publish it as-is.

I'm also working on a presentation slide for S3 conference http://www.hpi.uni-potsdam.de/hirschfeld/s3/s3-10/ to show it. I'm writing random thoughts about the compiler here so that I will compile them to a thread of talk.

I've already written the motivation on the paper (perhaps I will paste the URL in a month) so I don't repeat it. But in short, I wanted make a tiny language which bootstraps and runs on Adobe Flash.

A tiny language and bootstrapping seem contradicting idea as bootstrapping requires various language functions which tends to be large. On the other hand, this is practically a nice constrain because it keeps the language from too simple or too fat. Choosing Scheme like language as a target is natural to me because I wanted to concentrate basic implementation technique instead of language design.

Well, as one reviewer of the paper said, this is not particularly surprising or dramatically different in comparison with previous systems in the area, but some of the stories from the compiler should interest you!

How I started the assembler

In the beginning I created the assembler. Honestly, I wanted to avoid the task because writing assembler seemed not quite an interesting job. But in that time, I couldn't find a nice AVM2 assembler that suite my project. So I've done it. In retrospect, this was not bad at all. I could understand what avm2overview.pdf (the AVM2 specification) said quite well, and I got self confidence.

I wrote my assembler in PLT-Scheme because Ian Piumarta's COLA (Tamacola was supposed to be written in COLA and Tamacola itself, I'll tell you this later) is not finished yet in that time and Duncan Mak, a friend of mine, recommend it. This was actually a good choice. This is my first Scheme application and PLT's good documentation helped me a lot.

An interesting part of PLT-Scheme was it encourages a functional programming style, even PLT doesn't suppport set-car! and set-cdr! in the default library. So it was natural that my assembler was written without side-effect except I/O. This is the first key of the development of the assembler. Unfortunately, because Tamarin doesn't support tail-recursion optimazion and Tamarin's stack size is small, I gave up to eliminate all side-effect later. But the implementation was pure functional up to the time, and it was quite clean.

Indeed, it had to be clean considering boot-strapping. I wanted to make the assembler run in my language itself even before enough debugging facility is not ready. If it were not clean, a tiny bug would cause a few days of debugging. I avoided the nightmare with a functional style and Test Driven Development.

Test Driven Development is the second key. I virtually wrote every test case for each function even if it looks silly. Scheme has a couple of options of testing frame work. I chose SRFI-78. It only report assertion failer only something happen, otherwise it keeps silence. I somewhat like this UNIX taste terse.

The third key was to write an assembler and a disassembler in a same time. It sounds like an unnecessary job because I only needed an assembler eventually. But I had to analyze an output from asc (an asembler in Adobe Flex) and learn how an ActionScript program was converted to the Tamarin byte-code. The disassembler was very helpful to read the byte-code as well as debugging. If output of the disassembler generates the original byte-code by the assembler, there is high chance that my imprementation is correct, unless my understanding is wrong.

The assembler is named ABCSX http://github.com/propella/abcsx and it was ported to Gauche, COLA, and Tamacola later. I ported it to Gauche because I was curious about portability of Scheme language.

I had realized there are many places where I could reduce code redundancy in the assembler. An assembler tends to include repetitive process, but some of them are not quite captured well by function abstraction. I would be effective to apply macro and domain specific language in those part. I didn't have tried to solve it yet, but I want to solve it later.

(to be continued)

Recent Tamarin and ABC tools

2010-01-29T12:30:00.000-08:00

Tamarin-central, the stable source tree of open source VM used by Adobe Flash, was updated last December (Dec 22 2009) after relatively long blank. The newer tree has faster VM and includes updated ABC assembler and disassembler. Especially those ABC utilities are quite useful to a binary hacker of AVM2.

Download latest Flex SDK

I found that neither Flex SDK 3.5 nor 4.0 stable build can compile abcdump. You need to download later version from the Download Flex site. Flex 4-Beta 2 (4.0.0.10485) works well. I would set the Flex directory to environment variable FLEX.

$ export FLEX=~/Downloads/flex_sdk_4.0.0.10485_mpl

Download and build Tamarin-central

Building procedure is well documented in Tamarin_Build_Documentation. Only my additional suggestion is to add --enable-debugger, it makes error messages easy to read, it helps you, really.

$ hg clone http://hg.mozilla.org/tamarin-central/
$ cd tamarin-central
$ mkdir objdir-release
$ cd objdir-release
$ python ../configure.py --enable-shell --enable-debugger
$ make
$ ./shell/avmshell -Dversion
shell 1.5 release-debugger build cyclone
features AVMSYSTEM_32BIT; ...

Build abcdump

There are various useful utilities in utils/ directory. Some utilizes are written in ActionScript, so you need to compile them to use. Abcdump, ABC disassembler, is one of such utilities.

$ cd ..
$ java -jar $FLEX/lib/asc.jar -import core/builtin.abc -import shell/shell_toplevel.abc utils/abcdump.as

core/builtin.abc and shell/shell_toplevel.abc are basic libraries provided by tamarin, you can use them to try to see how abcdump works. Note that you need to separate abc file names with --, otherwise arguments are processed by avmshell instead of abcdump.

$ ./objdir-release/shell/avmshell ./utils/abcdump.abc -- core/builtin.abc 
// magic 2e0010
// Cpool numbers size 158 0 %
...

I recommend you to make a tiny shell script to ease such a complicated command line.

#!/bin/sh
~/tmp/tamarin-central/objdir-release/shell/avmshell ~/tmp/tamarin-central/utils/abcdump.abc -- $@

How to use abcasm

Abcasm is a ABC assembler. It is written in java and shell script, so you don't need to compile to try it. utils/abcasm/test/ directory includes various interesting sample programs. You can test them easily and quickly.

$ cd utils/abcasm/
$ ./abcasm.sh test/hello.abs
test/hello.abs
$ ../../objdir-release/shell/avmshell test/hello.abc
Hello, world

Wooden Half Adder Video

2009-12-18T06:12:00.000-08:00

*UPDATED* New half adder video is released!

A group exhibition "First Passage"

2009-11-28T22:30:00.000-08:00

I'm going to Osaka in the chiristmas season to join the exhibition. This is a kind of "college reunion show" and all of artists were students of Prof. Hitoshi Nomura. I'm very excited to see old friends of the sculpture course. Now I'm quite busy to build a new version of wooden half adder. This is my first exhibition since 1999 anyway. Time is running so fast.

First Passage

ARTCOURT Gallery Tenmabashi 1-8-5, Kita-ku, Osaka 530-0042 Japan

Hitoshi Nomura, Koji Nishimatsu, Kawai+Okamura, Osamu Kokufu, Tamaki Nagatani, Takashi Yamamiya, Yui Ishiyama, Kohei Nawa, Yasutaka Yamada, Yoshihiko Maeda, Ichiro Okada, Takao Machiba, Kenta Ogaya, Takayuki Okamoto, SHINCHIKA, Yasuyoshi Uchiyama and Taro Okumura

December 15 - 26, 2009
Gallery Hours: 11:00 - 19:00 ( - 17:00 on Saturdays) Closed on Sundays

Event : Artists Talk Show
Saturday, Dec.19, 14:00 - 15:30 (*15:30 - 17:00 Reception)

Wooden half adder

2009-11-05T22:21:00.000-08:00

Since I encountered an essay by A. K. Dewdney over ten years ago, somewhat I had been obsessed about mechanical implementation of boolean logic. The essay introduced ideas of kinetic binary calculator as an imaginary story of ancient civilization. This fantastic tale attracted me, and my theme at the art college became how to make a computer made from wood.

Last year, my boss gave me a Dewdney's book "The Tinkertoy Computer and Other Machinations", which includes the essay, and my enthusiasm came back. I made a couple of prototypes to realize boolean logic without electronics. This is the latest attempt.

I chose a half adder as the motive because half adders are used everywhere in a computer hardware, and it seems nice complexity for the project. I thought this is the best example to show the idea that computer's concept is independent with hardware. In other words, I wanted to prove that computer is purely conceptual entity and you can make it from any material besides electronics parts.

I first built a wooden AND operator. Can you tell how this toy work? The top two holes show the input, and the center hole shows the answer. Background board shows two kind of state T (True) and F (False). Because of gravity, initial state is T AND T = T, once you pull one of bars up, the center piece is also pulled up, and the answer becomes F. This is how AND works.

Basically, any boolean logic can be possible with AND, and another operator NOT (even OR can be constructed from AND and NOT). Let's think about a half adder. A half adder is a primitive adder and you can only add two bits. There are only four positions.

0 + 0 = 0 0
0 + 1 = 0 1
1 + 0 = 0 1
1 + 1 = 1 0

Let the inputs A and B, and the outputs S (sum, the low bit) and C (carry, the high bit). A half adder can be shown two boolean equations from AND(∧) and NOT(¬).

C = A ∧ B
S = A xor B = ¬(A ∧ B) ∧ ¬(¬ A ∧ ¬B)

Finally, I constructed parts along with this formula. I used gears as NOT operator.


1 + 1 = 1 0	1 + 0 = 0 1	0 + 1 = 0 1	0 + 0 = 0 0

An Assembler for AVM2 using S-Expression

2009-10-04T23:28:00.000-07:00

These days, I have been writing a documentation about my latest AVM2 assembler. Because it took very long time, I tempted to copy it here to make my blog seem substantial.

Overview

ABCSX is an assembler and disassembler for the ActionScript Virtual Machine 2 (AVM2) [1] and the ActionScript Byte Code (ABC). It runs on Cola/Amino language or PLT-Scheme. The syntax consists of s-expressions and a program can be constructed with normal list operations in Scheme like language. The goal of this utility is to build a high level language compiler for Adobe Flash Player. To get the idea, "Hello World!" programs for both ABCSX and abcasm (a standard assembler utility consisted in the AVM2 source tree [4]) are shown.

;;;; A "Hello World!" program in ABCSX ASM-form
(asm
 (method
  (((signature
     ((return_type *) (param_type ()) (name "hello")
      (flags 0) (options ()) (param_names ())))
    (code
     ((getlocal 0)
      (pushscope)
      (findpropstrict ((package "") "print"))
      (pushstring "Hello, World!!")
      (callproperty ((package "") "print") 1)
      (returnvoid))))))
 (script (((init (method 0)) (trait ())))))

// A "Hello world World!" program in abcasm
function hello():*
{
    getlocal 0
    pushscope
    findpropstrict print
    pushstring "Hello, World!!"
    callproperty print (1)
    returnvoid
}

Although a program written in abcasm syntax is more concise than ABCSX, the semantics is rather ambiguous. For example, in spite of each symbol name in ABC belongs to namespace(s), the syntax of abcasm doesn't describe it clearly. In this case, "print" is implicitly interpreted to a Multiple Namespace Name with a namespace set including PackageNamespace with no name. In case of ABCSX, it is explicitly represented as PackageNamespace with no name by ((package "") "print"). This implicit behavior might be useful for writing a program by hand, but not necessary for a machine generated code. ABCSX rather takes a direction toward verbose but unambiguous style.

ABCSX offers two forms of syntax. ASM-form is higher level syntax introduced above. ABC-form is identical to an abstract syntax tree of ABC binary file. This is useful when exact behavior is need to know while debug.

;;;; A "Hello World!" program in ABCSX ABC-form
(abc
 (minor_version 16)
 (major_version 46)
 (constant_pool
  ((integer ())
   (uinteger ())
   (double ())
   (string ("hello" "" "print" "Hello, World!!"))
   (namespace ((package (string 2))))
   (ns_set ())
   (multiname (((namespace 1) (string 3))))))
 (method (((return_type (multiname 0)) (param_type ())
           (name (string 1)) (flags 0) (options ()) (param_names ()))))
 (metadata ())
 (instance ())
 (class ())
 (script (((init (method 0)) (trait ()))))
 (method_body
  (((method 0) (max_stack 2) (local_count 1)
               (init_scope_depth 0) (max_scope_depth 1)
    (code
     ((getlocal 0)
      (pushscope)
      (findpropstrict (multiname 1))
      (pushstring (string 4))
      (callproperty (multiname 1) 1)
      (returnvoid)))
    (exception ())
    (trait ())))))

Using ASM-form, a compiler writer doesn't have to care about building a constant pool, or code hint information (AVM2 requires a frame information like stack size and register size used in a code).

Background

One of goals of the STEPS project [3] and COLA programming language is to provide full control of computer environment from application level to machine language level, so that users could experiment and design their own programming language best fit to their task. It also will be used as a basis of next generation of EToys programming environment for kids.

We chose Adobe Flash Player as one of platforms of the system because of its popularity and usability. Using Flash's virtual machine on a web browser, we could deliver our programming environment without concerning about installation or security issue.

AVM2 has some disadvantages compared to Java VM. AVM2 lacks multi task support, and its dynamic dispatching function is relatively slow. But the startup speed and memory footage are good, and these aspects are essential to casual users. Especially AVM2 will be good platform to implement EToys.

ABCSX is designed to be a back end module for COLA, command line assembler / disassembler, and a Scheme library. While it is a part of COLA/ABC compiler, it also can be used as a command line tool to examine and debug ABC binary file.

Usage

Command line tool

A version of ABCSX is publicly available on the github repository [2]. It includes command line tools run on PLT-Scheme. There are also example programs at examples/ directory. The assembler and disassembler use same file format and the assembler asm.ss can read an output file generated by disassembler dump.ss.

asm.ss filename.sx: Generate an ABC binary file from ASM-form or ABC-form. The output file name is filename.sx.abc.
dump.ss [-abc] filename.abc: Disassemble an ABC binary file. The output is printed to stdout. If -abc option is specified, ABC-form is chosen as output format.
runasm.sh filename.sx: Assemble ASM-form or ABC-form and execute it by avmshell. It requires avmshell installed. Avmshell is included in Tamarin VM's source tree [4].
swf_abc.erl width height classname abcfile.abc: A helper program to generate a flash file from an abc file. It requires Erlang.

Function

(write-asm list port) procedure: Assemble ASM- or ABC-form to a binary stream.
(read-asm port) procedure: Disassemble a binary stream to ASM-form.
(from-asm list) procedure: Convert ASM-form to ABC-form. This is a part of process of assemble. Each literal value is replaced to a reference, and a constant pool is created
(to-asm list) procedure: Convert ABC-form to ASM-form. This is a part of process of disassemble. Each constant reference in the ABC-form is replaced to a literal value based on the constant pool.

Data Type

ABC's data is expressed as scheme expression in ABCSX. In ASM-form, data conversion has subtle context dependency in code-subsection.

integer - An integer value in Scheme is converted to ABC integer value depend on the context.
- int (s32) - In code-subsection, an integer is converted to a signed 32 bit integer if the opcode requires integer e.g. pushint.
- uint (u32) - In code-subsection, an integer is converted to a unsigned 32 bit integer if the opcode requires integer e.g. pushuint.
- u30 - An integer is converted to a unsigned 30 bit integer in ABC anywhere else.
double (d64) - A floating point number value is converted to a 64-bit double precision IEEE 754 value.
string - A string is converted a string value in ABC.
namespace - Some list expressions are converted to namespace values in ABC. The format is (kind string). For example, (package "org.vpri") is converted to a package namespace named "org.vpri".
- Namespace - (ns string) is converted to Namespace
- PackageNamespace - (package string) is converted to PackageNamespace
- PackageInternalNs - (internal string) is converted to PackageInternalNs
- ProtectedNamespace - (protected string) is converted to ProtectedNamespace
- ExplicitNamespace - (explicit string) is converted to ExplicitNamespace
- StaticProtectedNs - (static string) is converted to StaticProtectedNs
- PrivateNs - (private string) is converted to PrivateNs
namespace set - A namespace set can not be described as a literal. Instead, it is declared in a constant pool of ns_set-section at first, and be made reference by index e.g. (ns_set 1).
multiname - Some list expressions are converted to multiname (symbol) in ABC.
- QName - (namespace string) is converted as QName e.g. ((package "flash.display") "Sprite"))
- RTQName - is not supported.
- RTQNameL - is not supported.
- Multiname - ((ns_set integer) string) is converted as a Multiname e.g. ((ns_set 1) "addChild")
- MultinameL - is not supported.

Syntax

The syntax of ASM-form is explained. ABCSX uses same symbol names as "ActionScript Virtual Machine 2 (AVM2) Overview" unless it is too strange. Especially, underline delimited names and capital names are derived from the document.

ASM-form

(asm [ns_set-section] method-section [metadata-section] [instance-section] [class-section] script-section)

ASM-form begins with a symbol asm, and contents are followed. ns_set-section, instance-section, and class-section are optional.

ns_set-section

(ns_set (ns_set namespace ...) ...)

Ns_set-section will be a part of constant pool, and it is only necessary if namespace set is used in other part of the ASM-form. You can not specify a namespace set directly as a literal, but you need to define it in ns_set-section and point it with the index number.

Ns_set-section begins with a symbol ns_set and a list of ns_set_info is followed. A ns_set_info begins with a symbol ns_set and it includes a list of namespaces. A namespace set is referred with one-based index by other part. For example, the first namespace set is referred as (ns_set 1).

method-section

(method (signature-subsection code-subsection) ...)

Method-section includes a list of pairs of signature and code. A method is referred by zero-based index. For example, the first method is referred as (method 0).

signature-subsection

(signature (return_type multiname) (param_type (multiname ...)) (name string) (flags integer) (options (option...)) (param_names (multiname ...)))

Signature-subsection describes method's signature. If * is specified at the return_type. It is treated as Any Type. A name entry is not used as a method name in a program. In a typical case, methods are explicitly bound to named slots in initialization code at script-section or object constructor.

code-subsection

(code (instructions...))

Code subsection describes a sequence of instruction code of the method. A label is specified as a symbol, and normal instruction is specified as a list as:

([offset-number] inst-name args ...)

offset-number is optional and used just as a place holder. It can be a integer or symbol _. ABCSX's disassembler put a byte offset number at this place, but the assembler ignores it.

metadata-section

(metadata (metadata_info ...))

Metadata-section describes a list of metadata entries.

instance-section

(instance (((name multiname) (super_name multiname) (flags integer) (interface (multiname ...)) (iinit method) (trait (trait_info ...)) ...)))

Instance-section describes a list of class definitions. Class members are defined by a list of trait_info.

class-section

(class (((cinit method) (trait (trait_info...))) ...))

Class-section describes a list of static members of class definition. The number of this list is same as instance-section, and each entry of class-section corresponds to instance-section. A definition consists of a class initializer and trait_info definitions.

script-section

(script (((init method) (trait (trait_info...))) ...))

Script-section defines a list of static functions. It is also used as a program's startup code. Once the virtual machine reads a program, the last entry of script-section is invoked. Each entry consists of a method reference and a list of trait_info. Trait_info is used as a function's environment.

trait_info

Trait_info defines a fixed property of an object, class, or method. ABCSX only supports Trait_Slot and Trait_Class.

Trait_Slot

((kind slot) (name multiname) (slot_id integer) (type_name multiname) (vindex integer) (vkind integer) (metadata (metadata_info...)))

Trait_Slot defines a named slot in the context.

Trait_Class

((kind class) (name multiname) (slot_id integer) (classi class) (metadata (metadata_info...)))

Trait_Class defines a named slot with a class in the context.

metadata_info

((name string) (items (((key string) (value string)) ...)))

Metadata_info defines an entry including arbitrary key/value pairs.

Current Status

Currently, only major elements in AVM2 are implemented.

All primitive data types are implemented.
75 instructions (about a half of the whole instruction set) are implemented.
Only QName (Qualified Name) and Multiname (Multiple Namespace Name) are implemented.
Optional parameters or parameter names are not implemented.
Trait_Method, Trait_Getter, Trait_Setter, Trait_Function, or Trait_Const are not implemented.
Exception is not implemented.

Example

As a complete example, A GUI version of "Hello World!" program is shown with commentary. This file is available at examples/textField.sx on the source tree.

(asm
 (ns_set
  ((ns_set (package "") (package "flash.text"))))

An ASM-form begins with a symbol asm, and a ns_set-section follows if necessary. This example declare one namespace set including package namespaces "" and "flash.text" as (ns_set 1). Ns_set's index number starts with 1 because this is a member of constant pool. Other kind of index number (method, class) starts with 0.

 (method
  (((signature ((return_type *) (param_type ()) (name "")
                (flags 0) (options ()) (param_names ())))
    (code
     ((returnvoid))))

The first method is referred as (method 0). It is used as a class initializer in the class-section, but nothing to do in this case.

   ((signature ((return_type *) (param_type ()) (name "")
                (flags 0) (options ()) (param_names ())))
    (code
     ((getlocal_0)
      (pushscope)
      (getlocal_0)
      (constructsuper 0)
      (findpropstrict ((ns_set 1) "TextField"))
      (constructprop ((package "flash.text") "TextField") 0)
      (coerce ((package "flash.text") "TextField"))
      (setlocal_1)
      (getlocal_1)
      (pushstring "Hello, World!")
      (setproperty ((package "") "text"))
      (findpropstrict ((package "") "addChild"))
      (getlocal_1)
      (callproperty ((package "") "addChild") 1)
      (pop)
      (returnvoid))))

The second method is later used in the instance-section as class Hello's constructor. It builds an instance of flash.text.TextField and set "Hello, World!" to the property named text. Finally, the text field is added to this (Hello) object.

   ((signature ((return_type *) (param_type ()) (name "")
                (flags 0) (options ()) (param_names ())))
    (code
     ((getlocal_0)
      (pushscope)
      (getscopeobject 0)
      (findpropstrict ((package "") "Object"))
      (getproperty ((package "") "Object"))
      (pushscope)
      (findpropstrict ((package "flash.display") "Sprite"))
      (getproperty ((package "flash.display") "Sprite"))
      (pushscope)
      (findpropstrict ((package "flash.display") "Sprite"))
      (getproperty ((package "flash.display") "Sprite"))
      (newclass 0)
      (popscope)
      (popscope)
      (initproperty ((package "") "Hello"))
      (returnvoid))))))

The third method is used as the startup script. It creates an environment and initialize a new class defined in instance-section and class-section by newclass instruction.

 (instance
  (((name ((package "") "Hello"))
    (super_name ((package "flash.display") "Sprite"))
    (flags 0)
    (interface ())
    (iinit (method 1))
    (trait ()))))
 (class (((cinit (method 0)) (trait ()))))

Instance-section and class section define classes. In this case, A class named Hello is defined as a subclass of flash.display.Sprite. When a SWF file is created from ABC file, a SymbolClass tag in the SWF creates association between a class name defined here and the main timeline of the SWF. In ABCSX tool set, script swf_abc.erl's third argument does this task.

 (script
  (((init (method 2))
    (trait
     (((kind class)
       (name ((package "") "Hello"))
       (slot_id 1)
       (classi (class 0))
       (metadata ()))))))))

Script-section defines the startup script and predefined named slot.

References

[1] ActionScript Virtual Machine 2 (AVM2) Overview. http://www.adobe.com/devnet/actionscript/articles/avm2overview.pdf
[2] ABCSX github repository. http://github.com/propella/abcsx
[3] Steps Toward the Reinvention of Programming (First Year Progress Report). http://www.vpri.org/pdf/tr2007008_steps.pdf
[4] Tamarin Project http://www.mozilla.org/projects/tamarin/

A simple boolean logic table generator in Javascript

2009-07-19T16:35:00.000-07:00

http://languagegame.org/pub/booltable/booltable.html

Boolean logic is a basic knowledge for all programmers. The idea is very simple and straightforward, yet still it often annoys you a lot. One of the reasons might be that boolean logic is not continuous behavior, and just a bit of changes makes completely different result. This is why it is hard to grab whole character of a boolean expression, I think.

This tiny program shows every possible state of a boolean expression to make sure your logic is correct. I wrote this to help me reading a textbook about logic, and to make programming nested condition statements easy.

One nice thing is "permlink" function. You can bookmark your favorite boolean expressions to play again. It works even in file URL as this is written entirely in Javascript. So feel free to download to your local disk.

Here are a couple of examples to link boolean tables:

Lazy list and Data flow in Squeak

2009-06-12T14:18:00.000-07:00

Introduction

I have been working on lazy list these days. A lazy list is a collection where each element is evaluated only when it's necessary. Lazy lists are very useful to express mathematical infinite sequence naturally like all natural numbers, entire prime numbers, or so.

Lazy lists are also practical when you deal with a large sequence of data. Imagine that you want to replace particular words in a large text, and print out just the first 100 characters. Without lazy list, you must either, 1) replace all words and take first 100 characters, or 2) count character position carefully each time when a word is replaced. 1) requires time, and 2) tend to lost modularity. With lazy lists, you can write such program as simple as former way, but performance is still reasonable.

Non-strict functional language like Haskell supports this feature in everywhere, but I was also interested in implementations of lazy lists in "normal" languages. Even though its long history, lazy lists have not been widely used. Lazy lists are sometimes called as "Stream", and indeed, Unix's pipe can be seen as a kind of lazy list. But it lacks generality. I wondered if I can find some traps to be solved when I implement it myself. Squeak is a powerful tool for the purpose because it has flexible runtime debugging support. And I believe most part of difficulty in the concept comes from difficulty of its runtime behavior.

SICP is a good tutorial and I found further discussions in SRFI-40 and SRFI-41. Ruby has good implementation similar to SRFI-41. And even Squeak has two implementations (LazyList and LazyCollections)! And here is my story.

Natural numbers

You can download a demo image from LazyList.zip or current source code from Collections-Lazy-tak.5.mcz. This source only uses basic functions in my home brewed image, but please let me know if it doesn't work in your Squeak image.

This is an expression to make natural numbers with LazyList.

list := LazyList nat.

I took advantage of Squeak tools, inspector and explore. If you inspect the variable "list" (select "inspect it" in the context menu), you will see internal structure of the list. And if you explore the list (select "explore it"), you see public "interface" of the list. I'll show you what I meant.

And when you open a list at the explorer (the right window), values of public accessors head and tail are shown, and also the inspector is updated. The first element 1 is made only because someone send head message. And you can get further numbers with clicking tail accessor. Note that while explorer triggers the list, inspector is inert so that you can freely observe internal behavior of the list.

You can do this by a program. Message head answers the first element of a list, and tail answers following list. This is same as CAR and CDR pair in lisp. Many other methods are defined e.g. take: returns a LazyList including first n elements, and contents convert LazyList to Array.

list head. "=> returns 1"
list tail. "=> returns LazyList .."

(list take: 10) contents. "=> #(1 2 3 4 5 6 7 8 9 10)"

And then, try next expression. You might know that select: is a message where a selected collection specified by a block as a condition is returned (this is "filter" in other language). But how possibly all natural numbers can be used with select:? Unlike other collection methods, it doesn't evaluate a condition block immediately. Instead, this method returns a new list which will be behave as a filter of natural numbers.

list select: [:e | e odd].

Define a new LazyList

Most useful way to construct a new lazy list would be using followedBy: method. The receiver becomes head of new list, and the argument becomes tail. You can use it with any receiver, and the argument must be a LazyList or block that returns a LazyList. LazyList nil is used to specify end of the list.

1 followedBy: (2 followedBy: (3 followedBy: LazyList nil))
"=> same as LazyList withAll: #(1 2 3)"

You can define natural numbers using followedBy: like this.

nat := 1 followedBy: [nat collect: [:e | e + 1]].

It seems very strange! But please be relax. The first element of the list is obviously 1. How about the second? collect: is sometimes called as map function in other languages, and

nat
collect: [:e | e + 1]

answers a LazyList with all element is just larger by 1 than nat itself. Therefore the sequence is generated by recursive fashion. How clever!

nat                                           = 1, ...
nat collect: [:e | e + 1]                     = 2, ...
nat = 1 followedBy: nat collect: [:e | e + 1] = 1, 2, ...
nat collect: [:e | e + 1]                     = 2, 3, ...
nat = 1 followedBy: nat collect: [:e | e + 1] = 1, 2, 3, ...
nat collect: [:e | e + 1]                     = 2, 3, 4, ...
nat = 1 followedBy: nat collect: [:e | e + 1] = 1, 2, 3, 4, ...

Squeak Smalltalk has some great heritage from APL. Many arithmetic functions are defined in collections. So does LazyList. There is a much simple definition of natural number.

nat := 1 followedBy: [nat + 1].

It reminds me an ugly assignment expression with side effect.

x := x + 1.

This is ugly because it looks like an equation but it is not. x is never equal to x + 1! Sometimes it confuses a beginner programmer. Still, it is too useful to ignore. We need some reasoning of such side effect to use safely. On the other hand, nat := 1 followedBy: [nat + 1] is similar enough to x := x + 1 yet, it is actually an equation. This is a significant coincidence, isn't it?

Iteration, Data flow, Lazy list, and Polymorphic functions

One of the problems in x := x + 1 is that value of x depends on the position in the source code. So you have to be careful about order of each expression. Are there any better ways? You can imagine more mathematical programming language to allow you to define a function as a equation.

x₁ = 1 --- some initial number
x_n = x_{n - 1} + 1

In this equations, it is clear that right hand side of x is older than left hand side of x. This is pure and easy to reason. But another problem occurs. In practice, the program requires whole history of xs because now a programmer can specify any old value like x_{n - 100}.

So I would add a restriction. What if only addition is allowed in subscript. In other words, what if only "future" value can be referred but "past" so that we can forget our unnecessary past conveniently (it is possible with nice GC).

x₁ = 1 --- some initial number
x_{n + 1} = x_n + 1

It makes the relationship between an iteration x := x + 1 and a lazy list more clear. This iteration can be converted a lazy list definition directly.

nat := 1 followedBy: [nat + 1].
      ~~~            ~~~~~~~~~
    x₁ = 1;        x_{n + 1} = x + 1

I know this discussion is still rough. You can find better discussion from Lucid. However, my theory is that data flow structures (mathematical iteration) can be expressed by lazy lists and polymorphic methods without special language.

Here is another example of definition of a lazy list in iterative way to compute Fibonnaci numbers. next is a synonym of tail.

fib := 1 followedBy: [1 followedBy: [fib + fib next]].

This corresponds next mathematical equations.

fib₁ = 1
fib₂ = 2
fib_{n + 2} = fib_n + fib_{n + 1}

Simulations

As a little bit more realistic example, I will show you a gravity simulation using lazy lists. In this demo, a global variable Y shows height of an object, the first position is 400, and it is increased by Speed. The initial value of Speed is 0 and the value is decreased by 1 while the hight Y is positive. If Y is negative, Speed is negated which means the object is bounced by the ground!

Y := 400 followedBy: [Y + Speed next].

Speed := 0 followedBy: [Y > 0 whenTrue: Speed - 1
                             whenFalse: Speed negated].

Morphic part of this demo used a special morph named ListPointer that worked as an interface between LazyList and Morphs.

I began this library as a replacement of Stream and OrderedCollection. But it turned out that combination of polymorphic methods and LazyList has unexpected cute property as data flow. I'm very happy about the fact! If you are interested in rather practical aspect of lazy lists, please check out List >> show method in this library. printString for LazyList is implemented this framework itself.

I already noticed that there are a lot of rooms I could improve it. The design was briefly along with SRFI-40, but I will include more ideas in SRFI-41 and Ruby's lazylist. I have ignored them so far simply because I am lazy!

A complete example of generating a sound in Flash with SampleDataEvent

2009-05-26T13:48:00.000-07:00

Flash version 10 has significant feature about audio, SampleDataEvent. Until this version, there was no way to generate a sound wave easily. Some people have tried to do that by rather magical way of how to make actionscript 3 play generated pcm wave data in Flash 9, however, now we have a straightforward API from Flash 10. I think this is a great improvement for hobby programmers who are interested in computer generated music. While it's shame from some perspective. Just this API finally brings us a point where old BASIC and PLAY statement could do a quarter centry ago!

You would find a lot of examples about SampleDataEvent, but some of those are wrong or uncompleted, since I think this API have been changed so often in beta version. And I wrote my version of simplest example for exercise.

There are a couple of tricks you might trap.

If you get an error message: "Flex error: type was not found or was not a compile-time constant: SampleDataEvent", you have to specify the Flash version to 10.0.0. Set -target-player option or Flex builder's Project's Properties - ActionScript Compiler - HTML Wrapper
Some documentation might use Event.SAMPLE_DATA, but this is not true.

Generated SWF file: http://languagegame.org/pub/SineSound.swf

A Prolog In Haskell

2009-04-28T02:51:00.000-07:00

Prolog In Haskell

I have been obsessed by Prolog language recent weeks. While I first learned Prolog long time ago, and actually I was attracted, I have never used it fluently because it's too hard to get familiar without any practical necessity. Although, there are a lot of interesting literatures which require certain knowledge of logic programming in computer science. So, I decided to do another approach; writing a Prolog interpreter to learn Prolog.

I chose Haskell as the implementation language because of its succinctness. I'm a beginner Haskell programmer, and I thought it was also a good opportunity to learn Prolog and Haskell same time! The starting point was a Prolog implementation in Hug98 distribution http://darcs.haskell.org/hugs98/demos/prolog/. I think this is a great Haskell program, but its too difficult to me. Rewriting it as my level would be a good exercise.

Data structures

Here is my version of Prolog in Haskell. Entire program is about 200+ lines. There is no cut operator but it has a list notation so that you can write [apple, orange | banana] stile literal. Let's take a look at the first part.

module Prolog
    (Term(..), Clause(..), w, s, cons,
     parse, parse',
     atom, variable, struct, list, nil, terms, arguments, term, clause, clauses, query,
     display,
     unify, unifyList, applyTerm, prove, rename, solveString) where

import Text.ParserCombinators.Parsec
import Data.Maybe
import Char

I used Parsec as a parser, and defined some data structures.

infix 6 :-
data Term    = Var String Int | Struct String [Term] deriving (Show, Eq)
data Clause  = Term :- [Term]                        deriving (Show, Eq)
data Command = Fact Clause | Query [Term] | ShowAll | Noop

type Rules = [Clause]
type Substitution = [(Term, Term)]

Term represents Prolog's term like "apple" or "father(abe, homer)". It can be a variable, or a structure. A variable has an index number which I used it later to distinct a same variable name in different contexts. A simple term like "apple" is also represented as a structure without elements like Struct "apple" [].

Clause is a Prolog rule like "mortal(X) :- man(X)". I stole a user defined operator constructor ":-" from original Hugs' Prolog to write a Haskell expression in Prolog like. So "mortal(X) :- man(X)" in Haskell expression becomes Struct "mortal" [Var "X" 0] :- [(Struct "man" [Var "X" 0])]. Well, it's not quite nice. Although the parser will provide better notation later, I have to use this expression when debugging the interpreter meanwhile. Its cumbersome. So I made up tiny utility functions to make Prolog data easier.

-- Utility constructors for debugging
w :: String -> Term
w s@(x:xs) | isUpper x = Var s 0
           | otherwise = Struct s []

s :: String -> [Term] -> Term
s n xs = Struct n xs

cons s cdr = (Struct "cons" [w s, cdr])

Using this functions, now "mortal(X) :- man(X)" is written as


s"mortal" [w"X"] :- [s"man" [w"X"]]

. It is much better, isn't it?

Unification

By the way, I like the word unification. It sounds peace and spiritual! Unification is one of two most peculiar concept in Prolog (another one is the control structure by depth first search). Unification is solving a logical equation. For example, the answer of "[X, orange] = [apple, Y]" must be X = apple, and Y = orange. It is almost same as variable binding in a normal programming language, but tricky part is that a direction is symmetry, so X = Y and Y = X is same meaning. How can it be possibly implemented?? Think about the data structure of the answer at first.

---- Unification ----

type Substitution = [(Term, Term)]
true = []

I used a list of tuples of terms, or an associate list to represent a substitution. For example, "X = apple, Y = orange" is represented as [(X, apple), (Y, orange)] (in actual Haskell code, [(w"X", w"apple"), (w"Y", w"orange")] ). A tuple of left hand side is always a variable name, and right hand side is any term, concrete value preferably. The goal of unification is making associations with variable and term. To make this process easier, "transitive" substitution is allowed. Think about an equation "X = Y, Y = banana". It is represented like [(X, Y), (Y, banana)], which is solved as X = banana, and Y = banana in apply function. Let's look at the implementation.

-- apply [(w"X", w"Y"), (w"Y", w"Z")] [(w"X"), (w"Y")] == [(w"Z"), (w"Z")]
apply :: Substitution -> [Term] -> [Term]
apply s ts = [applyTerm s t | t <- ts]

applyTerm [] (Var y n)                                  = Var y n
applyTerm ((Var x i, t):s) (Var y j) | x == y && i == j = applyTerm s t
                                     | otherwise        = applyTerm s (Var y j)
applyTerm s (Struct n ts)                               = Struct n (apply s ts)

The function apply substitutes a variable name of its value. To support transitive apply, applyTerm is called recursively if the value is also a variable. But it can solve only one way. Think about opposite case "Y = banana, X = Y". Apply can't find the fact X = banana because "Y = banana" is appeared before. So what I should do is applying X = Y before adding the substitution.

	Equation	Substitution(solution)
1	Y = banana, X = Y
2	X = Y	Y = banana (append)
3	X = banana (apply: Y = banana)	Y = banana
4		Y = banana, X = banana (append)

I suppose that this two fold way solve all of logical equation. Apply is always needed before append it to the solution. Actual source implementation seems to be complicated because there are cases where a variable can appears any side, and sometimes there is no solution. To represent no-answer case, a Maybe monad is used. So there are variations such as;

No Answer : Nothing
Always true (like apple = apple) : Just true (true is a synonym of empty list [])
Substitution found : Just [X = some answer...]

-- unify (w"X") (w"apple") == Just [(w"X", w"apple")]
unify :: Term -> Term -> Maybe Substitution
unify (Var x n) (Var y m) = Just [(Var x n, Var y m)]
unify (Var x n)      y    = Just [(Var x n,       y)]
unify      x    (Var y m) = Just [(Var y m,       x)]
unify (Struct a xs) (Struct b ys)
      | a == b = unifyList xs ys
      | otherwise   = Nothing

unifyList :: [Term] -> [Term] -> Maybe Substitution
unifyList [] [] = Just true
unifyList [] _ = Nothing
unifyList _ [] = Nothing
unifyList (x:xs) (y:ys) = do s <- unify x y
                             s' <- unifyList (apply s xs) (apply s ys)
                             return (s ++ s')

Note that I just use append (++) to add a new substation in unifyList. But if you design carefully, recursive apply is not necessary. Using something like a map is a better idea.

Solver

As a programming language, Prolog is unique as it has no explicit control structure. Instead, a Prolog program can be seen as a big nested if then else statement. This find and branch functions are implemented of this behavior. While unification is a technique of how to solve a equation, solver deals with when each equation should be solved. There are two most important concepts to understand control structures in Prolog.

Goals (AND relationship) : are terms which should be solved.
Branches (OR relationship) : are options which might have a solution

A proof's fate is decided by branch function, branch function returns a list of goals (with corresponding substitutions). If the list is empty, this branch is failed. If the list includes empty goal, it is actually succeed because empty goal means that it is unified against a fact like "food(apple).". Well, is it complicated?

If branch returns [] <--- this is failed
If branch returns [ [some goals] [some goals] [(empty)] <--- this is succeed! ]

The goal of find function is traverse all of goals to find whether if it is true or false.

---- Solver ----

prove :: Rules -> [Term] -> [Substitution]
prove rules goals = find rules 1 goals

-- Depth first search
-- find (parse' clauses "p(X):-q(X). q(a).") 1 [parse' term "p(X)"]
find :: Rules -> Int -> [Term] -> [Substitution]
find rules i [] = [true]
find rules i goals = do let rules' = rename rules i
                        (s, goals') <- branch rules' goals
                        solution <- find rules (i + 1) goals'
                        return (s ++ solution)

-- Find next branches. A branch is a pair of substitution and next goals.
-- branch (parse' clauses "n(z). n(s(X)):-n(X).") (parse' query "?-n(X).")
branch :: Rules -> [Term] -> [(Substitution, [Term])]
branch rules (goal:goals) = do head :- body <- rules
                               s <- maybeToList (unify goal head)
                               return (s, apply s (body ++ goals))

Find function has an argument for index number to show the depth of the tree. This number is used to rename all variables used in whole rules. This is necessary because same variable name in different clauses are actually represented different variables.

-- Rename all variables in the rules to split namespaces.
rename :: Rules -> Int -> Rules
rename rules i = [ renameVar head :- renameVars body | head :- body <- rules]
    where renameVar (Var s _)     = Var s i
          renameVar (Struct s ts) = Struct s (renameVars ts)
          renameVars ts           = [renameVar t | t <- ts]

I have only explained evaluator part of the REPL, but still there are reader, printer, and loop. You can browse and download whole source code from http://github.com/propella/prolog/tree. Someday I might write some of interesting topics in the program...

Learning Erlang and Adobe Flash format same time part 2

2009-04-05T00:01:00.000-07:00

I have already written most important part about SWF format and Erlang's bit syntax at previous post. But just for the record, I take a note of miscellaneous knowledge to construct an interesting Flash shape. This entire note doesn't have any practical value because there are already many other useful libraries to build a swf file. But some of historical esoteric magical peculiar internal might interests you.

In the previous example, I only made three control tags:

End (Tag type = 0)
ShowFrame (Tag type = 1)
SetBackgroundColor (Tag type = 9)

Here, I introduce you two additional tags,

DefineShape (Tag type = 2) defines a vector graphics, and
PlaceObject2 (Tag type = 26) shows the graphics to an actual screen.

I designed new data structure to represent a SWF shape. While spec/0 is almost same as previous program, I add sample_shape/0 to define DefineShape.

% an example shape.
spec() ->
    {{frame_size, 6000, 3000},
     {frame_rate, 16},
     {frame_count, 160},
     [{set_background_color, {rgb, 240, 250, 250}},
      sample_shape(),
      {place_object2, 1},
      {show_frame},
      {end_tag}]}.

sample_shape() ->
    {define_shape,
     {shape_id, 1},
     {bounds, {rectangle, 2000, 3000, 1000, 2000}},
     {shapes, {{fills, [{solid_fill, {rgb, 50, 200, 50}}]},
        {lines, [{line_style, 100, {rgb, 100, 255, 100}},
   {line_style, 100, {rgb, 50, 150, 50}}]},
        {shape_records,
  [{style_change, [{move_to, 2000, 2000}]},
   {style_change, [{line_style, 1},
     {fill_style0, 1}]},
   {straight_edge, 0, -1000},
   {straight_edge, 1000, 0},
   {style_change, [{line_style, 2},
     {fill_style0, 1}]},
   {straight_edge, 0, 1000},
   {straight_edge, -1000, 0},
   {end_shape}]
        }}}}.

It looks complicated though, it is actually very straightforward. A DefineShape has a shape ID (or character ID), its bounding box, and actual shapes. "shape_id", "bounds", and "shapes" tuples at sample_shape/0 denote these information. Shape ID is used later for a reference by PlaceObject2.

A Shapes part consists of further sub elements, fills, lines and shape records. Every style information are defined in advance, and referenced with corresponding ID. In this case, one solid fill and two line styles are defined. After styles are defined, shape records are followed. A shape record is the most interesting and complicated part. I'll draw a moss green rectangle in 1000 x 1000 twips (20 twips = 1 pixel) with light and dark green borders in a pseudo 3-D style.

This is an actual implementation of DefineShape and PlaceObject2. I'm afraid it seems to be complicated, but it is just a direct translation from the specification. Thanks to Erlang's pattern matching syntax, It is easy to add a new tag by putting a new pattern. But sometimes distinction between ; (semi colon is used between patterns in a function) and . (dot is used after a function definition) annoys me.

tag({define_shape,
     {shape_id, ShapeId},
     {bounds, Bounds},
     {shapes, ShapesWithStyle}}) ->
    BoundsBits = rectangle(Bounds),
    Shapes = shape_with_style(ShapesWithStyle),
    record_header_body(2, list_to_binary([<<ShapeId:16/unsigned-little>>, BoundsBits, Shapes]));

tag({place_object2, CharacterId}) -> 
    PlaceFlagHasClipActions = 0,
    PlaceFlagHasClipDepth = 0,
    PlaceFlagHasName = 0,
    PlaceFlagHasRatio = 0,
    PlaceFlagHasColorTransform = 0,
    PlaceFlagHasMatrix = 0,
    PlaceFlagHasCharacter = 1,
    PlaceFlagMove = 0,
    Depth = 1,
    record_header_body(26, <<PlaceFlagHasClipActions:1,
       PlaceFlagHasClipDepth:1,
       PlaceFlagHasName:1,
       PlaceFlagHasRatio:1,
       PlaceFlagHasColorTransform:1,
       PlaceFlagHasMatrix:1,
       PlaceFlagHasCharacter:1,
       PlaceFlagMove:1,
       Depth:16/unsigned-little,
       CharacterId:16/unsigned-little>>).

shape_with_style/1 is used to construct SHAPEWITHSTYLE structure in the specification.

shape_with_style({{fills, Fills}, {lines, Lines}, {shape_records, Shapes}}) ->
    FillBits = nbits_unsigned([length(Fills)]),
    LineBits = nbits_unsigned([length(Lines)]),
    ShapeRecords = shape_records({shape_records, Shapes}, FillBits, LineBits),
    [fill_style_array({fills, Fills}),
     line_style_array({lines, Lines}),
     <<FillBits:4,
      LineBits:4>>,
     ShapeRecords].

The tricky part is FillBits and LineBits. As I have already mentioned when explaining a rectangle format in previous post, the SWF format tries to shorten a file size by all means, bit by bit. FillBits defines maximum bit size needed by fill style id. For example, if you use only one fill style, one is enough for FillBits. But if you use seven, FillBits is three (because seven is 111 as a binary). After FillBits and LineBits are calculated, they are needed in next helper function shape_records/3. But before I show you shape_records/3, some simple functions to construct styles are followed.

fill_style_array({fills, FillStyleArray}) ->
    FillStyleCount = length(FillStyleArray),
    [<<FillStyleCount:8>>, [fill_style(X) || X <- FillStyleArray]].

fill_style({solid_fill, RGB}) -> [<<16#00:8>>, rgb(RGB)].

line_style_array({lines, LineStyleArray}) ->
    LineStyleCount = length(LineStyleArray),
    [<<LineStyleCount:8>>, [line_style(X) || X <- LineStyleArray]].

line_style({line_style, Width, RGB}) ->
    [<<Width:16/unsigned-little>>, rgb(RGB)].

I think these are straightforward. Perhaps, next is the most tricky part of the SWF shape. Shape records include a number of records, each record represents style change or position change or so. While a rectangle format must to be byte aligned, each record must not be aligned, but whole shape records including them must be aligned! This fact is described in the specification very obscure way, so I didn't find out what to do unless many try-and-errors. Although, I might misunderstood what is a shape record...

Each individual shape record is byte-aligned within an array of shape records; one shape record is padded to a byte boundary before the next shape record begins. --- SWF File Format Specification Version 10

To concatenate many bit strings into one, and align whole bits, I made two functions padding/1 and concat_bits/1 (padding/1 is same one as previous post). It is likely that concatenating bitstrings exists already, but I couldn't find out from the library.

shape_records({shape_records, ShapeRecords}, FillBits, LineBits) ->
    padding(concat_bit([shape_record(X, FillBits, LineBits) || X <- ShapeRecords])).

padding(Bits) ->
    Padding = 8 - bit_size(Bits) rem 8,
    <<Bits/bitstring, 0:Padding>>.

concat_bit(XS) -> lists:foldl(fun(X, Y) -> <<Y/bitstring, X/bitstring>> end, <<>>, XS).

Rest of the source is definitions of each record. I omitted a lot of features to the explanation purpose. EndShapeRecord and StraightEdgeRecord are quite simple.

shape_record({end_shape}, _, _) -> <<0:1, 0:5>>;

shape_record({straight_edge, DeltaX, DeltaY}, _, _) ->
    NumBits = nbits_signed([DeltaX, DeltaY]),
    GeneralLineFlag = 1,
    <<1:1, 1:1, (NumBits - 2):4, GeneralLineFlag:1, DeltaX:NumBits, DeltaY:NumBits>>;

But StyleChangeRecord is the second tricky and daunting part in spite of Erlang's succinct syntax. A part of reasons of the complexity is that StyleChangeRecord allows to define a couple of operations at once optionally; line style, fill style, and pen move. So we need some data structure to represent such option.

In object oriented languages, a dictionary or a map is commonly used while an associated list is popular in functional languages. In Erlang, a list of tuples might be enough. lists:keysearch/3 find a tuple in a list where first argument is matched, so we can use it as a dictionary like structure. In our example, [{line_style, 1}, {fill_style0, 1}] means we want to change line style and fill style same time. and lists:keysearch(line_style, 1, ChangeSpec) detects a line style change option.

shape_record({style_change, ChangeSpec}, FillBits, LineBits) ->
    TypeFlag = 0,
    StateNewStyles = 0,
    StateFillStyle1 = 0,

    {StateLineStyle, LineStyleData} =
 case lists:keysearch(line_style, 1, ChangeSpec) of
     {value, {_, LineStyle}} -> {1, <<LineStyle:LineBits>>};
     false -> {0, <<>>} end,

    {StateFillStyle0, FillStyle0Data} =
 case lists:keysearch(fill_style0, 1, ChangeSpec) of
     {value, {_, FillStyle0}} -> {1, <<FillStyle0:FillBits>>};
     false -> {0, <<>>} end,

    {StateMoveTo, MoveData} =
 case lists:keysearch(move_to, 1, ChangeSpec) of
     {value, {_, MoveDeltaX, MoveDeltaY}} ->
  MoveBits = nbits_signed([MoveDeltaX, MoveDeltaY]),
  {1, <<MoveBits:5,
       MoveDeltaX:MoveBits/signed,
       MoveDeltaY:MoveBits/signed>>};
     false -> {0, <<>>} end,

    <<TypeFlag:1,
     StateNewStyles:1,
     StateLineStyle:1,
     StateFillStyle1:1,
     StateFillStyle0:1,
     StateMoveTo:1,
     MoveData/bitstring,
     FillStyle0Data/bitstring,
     LineStyleData/bitstring>>.

I uploaded final program http://languagegame.org/pub/swf.erl.

Learning Erlang and Adobe Flash format same time

2009-03-27T13:40:00.000-07:00

Since Luke Gorrie encouraged me to learn Erlang, I have been playing with a bit with the language. One of the most attracting points is bit syntax. When you want a byte array with 65, 66, 67. I can write it as <<65,66,67>>. Further more, When you want a bit pattern of 4 in four bits, 1 in four bits and 0x4243 in 16 bits, you may say <<4:4, 1:4, 16#4243:16>>. What a cool!

$ erl
Eshell V5.6.4  (abort with ^G)
1> <<65,66,67>>.  % numbers are printed with ASCII
<<"ABC">>
2> <<4:4, 1:4, 16#4243:16>>.
<<"ABC">>

I was soon tempted to build a meaningful binary data with this syntax. What does suit? Adobe Flash format would be the weirdest and coolest. It's weirdest because of very bits oriented taste by perhaps historical reasons. It's coolest because it's popular. I want to show you how Erlang is powerful by writing complicated SWF format in the sophisticated syntax. Fortunately, the SWF specification is now free and everybody can write a flash editor by themselves.

First of all, I chose scripting style because I'm too lazy to wait for a compiler. And I start a shell bang line.

#!/usr/bin/env escript
%% -*- erlang -*-
%%! debug verbose

I don't understand how it works, but I just copied from the manual. And then, I have to decide what my first flash file looks like. I'm modest enough to satisfy just a salmon colored window. This is a spec file representing my flash.

% an example flash contents.
spec() ->
    {{frame_size, 2000, 3000},
     {frame_rate, 10},
     {frame_count, 10},
     [{set_background_color, {rgb, 250, 100, 100}},
      {show_frame},
      {end_tag}]}.

This is already reflected by the structure of a flash file. A flash file starts with meta data of file size, frame rate, and frame count. And a bunch of tagged data are followed. Tagged data is the key of extensibility of the flash format. You can embed various kinds of media as various type of tags in a file. In Erlang, fixed sized data is represented as {tuples}, and variable sized data is [lists]. I write each element as {key, arg1, arg2, ...} style, and tags are represented as a list. You might understand each tuple's meaning. You can change the color or size if you like.

Rest of the program reads this data and write a SWF file. I'll explain the program in a top down fashion. The simple main function opens a file and write something. Actual data is created by swf/0 (name/arity is Erlang's way to point to a function) as a sequence of binary.

main(_) ->
    Filename = "simple.swf",
    {ok, S} = file:open(Filename, [binary, write]),
    file:write(S, swf(spec())),
    file:close(S).

Now interesting part begins. If you read Chapter 2: SWF Structure Summary in the SWF specification, you may find that the Erlang program is much more concise and easier to understand than long description in the specification (although, I admit that I omitted some details). Especially, a bit syntax part is obvious, "FWS" is three byte characters, 'Version:8/unsigned-little' is a 8 bits integer for version number, and 'FileLength:32/unsigned-little' follows as a 32 bits little endian unsigned integer for file length. 'Rest/binary' looks tricky, but it means that another binary data Rest is embedded here.

% SWF Structure

swf(Spec) ->
    Version = 10,
    Rest = swf_rest(Spec),
    FileLength = size(Rest) + 8,
    <<"FWS",
     Version:8/unsigned-little,
     FileLength:32/unsigned-little,
     Rest/binary>>.

Because I needed to know the file size in advance, I had to split the main part in two. Swf_rest/1 composes every elements in data, and swf/1 mesures its file size and puts to the top. To pass my sample data to swf_rest/1, I use patterns. Thanks to to tuples and patterns, I don't have to remember an order of arguments. It looks like keyword arguments and more generic. Note that words begins with a lower case like 'frame_size' are symbols and words begins with a capital like 'Width' are variables in Erlang, in my case, 2000 is set to Width. In SWF file, a special unit 'twips' is used, 20 twips are a pixel.

A list comprehensions is another useful feature. I only used it as a shorthand of "map" function like [tag(X) || X <- Tags], which means apply tag/1 function to each element in Tags, but it saved many typing.

swf_rest({{frame_size, Width, Height},
   {frame_rate, FrameRate},
   {frame_count, FrameCount},
   Tags}) ->
    FrameSize = rectangle({rectangle, 0, Width, 0, Height}),
    FrameRateData = fixed8dot8(FrameRate),
    TagsData = list_to_binary([tag(X) || X <- Tags]),
    <<FrameSize/binary,
     FrameRateData/binary,
     FrameCount:16/unsigned-little,
     TagsData/binary>>.

Let's take a look at some detail of data type used in SWF. Color is simple, it is just an array of three bytes for Red, Green, and Blue. The description of 16-bit 8.8 fixed-point number in the specification looks very complicated, but it just said that you have to put the integer part last.

% Basic Data Types

rgb({rgb, R, G, B}) -> <<R:8, G:8, B:8>>.

fixed8dot8(N)->
    IntegerPart = trunc(N),
    SmallPart = trunc((N - IntegerPart) / 1 * 256),
    <<SmallPart:8, IntegerPart:8>>.

This is my favorite part. Rectangles are stored as very bit-oriented way. First, you must get a enough bit size to store each element (Nbits). Second, four elements are stored in this bit size. Third, entire bits should be byte-aligned. This kind of tasks are very tricky in other languages, it require a lots of shifts, ands, ors, etc. But Erlang's bit syntax helps me a lot.

I wrote two (actually three) helper functions to make rectangle/1 simpler. nbits_signed/1 calculate enough bit size for a list of signed integers, and padding/1 fills zeros to align byte width. '/bitstring' is same as '/binary', but you can use it for any odd size of bits data while binary works only byte arrays.

rectangle({rectangle, Xmin, Xmax, Ymin, Ymax}) ->
    Nbits = nbits_signed([Xmin, Xmax, Ymin, Ymax]),
    padding(<< Nbits:5,
      Xmin:Nbits/signed-big,
      Xmax:Nbits/signed-big,
      Ymin:Nbits/signed-big,
      Ymax:Nbits/signed-big>>).

nbits_unsigned(XS) -> % Necessary bits size for a list of integer values.
    Max = lists:max([abs(X) || X <- XS]),
    trunc(math:log(Max) / math:log(2)) + 1.
nbits_signed(XS) -> nbits_unsigned(XS) + 1.

padding(Bits) ->
    Padding = 8 - bit_size(Bits) rem 8,
    <<Bits/bitstring, 0:Padding>>.

Tags are the most important data structure in SWF. There are short and long tags. Short one is used for 62 bytes of data or less. Because tag code and length is packed in one place, you can save precious 16 bits space with short tag! Tricky part is here. In short tag, type code and length are stored respectively 10 bits and 6 bits, however, you have to save it as whole 16 bits integer in a little endian.

Big endian
|Type               |Length     |
|9|8|7|6|5|4|3|2|1|0|5|4|3|2|1|0|

Little endian
|Typ|Length     |Type           |
|1|0|5|4|3|2|1|0|9|8|7|6|5|4|3|2|

What a crazy?! (actually, it is not so strange than it looks if you see carefully. Hint: bits runs right to left, bytes runs left to right). Even these funny endianess conversion is handled easily in Erlang because bit syntax can be used in patterns (input).

% Tag Format

record_header_body(Type, Body) -> record_header_body(Type, Body, size(Body)).

record_header_body(Type, Body, Length) when Length < 63 ->
     <<TagCodeAndLength:16/unsigned-big>> = <<Type:10, Length:6>>,
    [<<TagCodeAndLength:16/unsigned-little>>, Body];

record_header_body(Type, Body, Length) ->
    <<TagCodeAndLength:16/unsigned-big>> = <<Type:10, 63:6>>,
    [<<TagCodeAndLength:16/unsigned-little>>,
     <<Length:32/unsigned-little>>,
     Body].

Finally, I'll show you the format of actual contents. Now I have only three tags for End (type=0), ShowFrame(type=1), and SetBackgroundColor(type=9). You might easily add new tags to support more convincing flash contents. Latest tag number is 91 for DefineFont4, see APPENDIX B Reverse index of tag values in the specification.

% Control Tags

tag({end_tag}) -> record_header_body(0, <<>>);
tag({show_frame}) -> record_header_body(1, <<>>);
tag({set_background_color, RGB}) -> record_header_body(9, rgb(RGB)).

I uploaded an entire program at swf_simple.erl Erlang is famous for its concurrent programming model, but I found another aspects like bit syntax and pattern matching are also attractive and useful to real application.

The SWF Specification: http://www.adobe.com/devnet/swf/pdf/swf_file_format_spec_v10.pdf

Inspy from Turtle Geometry

2008-10-06T20:37:00.000-07:00

Turtle Geometry by Harold Abelson and Andrea diSessa has a lot of interesting examples of turtle graphics. Especially I was fascinated "inspy" program. The basic structure of inspy is same as spiral. But instead of the side, value of the angle is changed in each step. I have explored a bit for a demo of Chalkboard project http://tinlizzie.org/chalkboard/#Inspi.

To make the study easier, I made a trick to connect an event handler and an inspy program. You can play with a dynamic version of inspy program with it (Please move the mouse on next rectangle). And you may find various patterns even from such simple program. Here I present stand alone version (Only Firefox and Safari are supported).

inspi = function(side, angle, inc) {
  for (var i = 0; i < 720; i++) {
    forward(side)
    right(angle + inc * i)
  }
}

inspi(15, mouseX(), mouseY())

Ostranenie - Crossover between Art and Programming

2008-09-23T00:11:00.000-07:00

It looks like that I have to say something at a camp next week. So I trumped up the most easy topic for me, the art. For me, the art is very common and familiar thing that I spent a lot of time to discuss such matter in my youth. And more importantly, it is somewhat strange topic for colleagues. I hope it will be fun.

One of the most important functions in the art is ostranenie. Ostranenie is a term coined by Russian critic Shklovsky, which means looking a familiar things in unfamiliar ways to find a unexpected side of the subject. For example, most surprising thing for a student who learns a sketch first time was there are no outlines in the world. Young kids often draw an outline to express the mom's face. But reality is that a human brain extracts an outline from a contrast of of light and shade.

Also a programmer often faces ostranenie. A human tends to think based on an intuition. But a computer only obeys logics, it never cares about human's common sense or intuition. The conflict often cause errors. But the errors give us precious ideas that we have never noticed. Human beings have developed common sense and intuition through the long history and culture, so it is very difficult to perceive a difference between logic and intuition without a computer.

We can regard the invention of computer in philosophy as the invention of photograph in the history of art. The invention of photograph provided an artist first different eyes beside human's. And it brought Impressionism in 19th. Likewise, we met another kind of brain for the first time by computers. No one knows what is born from there because the history of computer is still young. But we can explore new directions in terms of ostranenie.

Chalkboard

2008-09-11T14:31:00.000-07:00

Chalkboard is my latest web authorizing tool for active essay. I have just uploaded it here http://tinlizzie.org/chalkboard/. I tested both Firefox and Safari. I know there are still a lot of UI / technical issues though, I am sure that basic feature is enough for my purpose, and I will keep it simple.

My biggest concern at now is supporting Internet Explorer. I thought it was not so difficult for such a simple web application. But actually it was horrible! and I couldn't solve yet. I might go bad direction though, I was really disappointed about different behavior of layout. One can say that it is unnecessary to support various platforms for such an experimental project. But I think it is important because I hope it will get real users finally.

When I had to decide a name of directory for this project. I named it "chalkboard". I felt the spell was too long as a directory name, but I'm comfortable about this naming now.

Here are features.

The screen have three areas, editor, play area, and transcript. You can write text or source code on the editor. Any text is executed as a JavaScript program with [do it] or [print it] command. [do it] command prints the answer on the transcript, and [print it] shows the answer on the point of cursor. The play area provides a free space to user. It is used any purpose, for example, you can make a drawing tool to attach canvas element on it. See http://tinlizzie.org/chalkboard/#Scribble.

The Chalkboard includes a bunch of pages. Each page consists text and source code. Sauce code is usually written on <pre> tag.

All source codes written on <pre> tag in a page are invoked with [run] from top to bottom. This is useful when you encounter unknown page, and want to try at first. <pre> tag is used when the project is called by includes function. See http://tinlizzie.org/chalkboard/#Polygon

You can make a new page to specify new name after hash mark of the URL like http://tinlizzie.org/chalkboard/#NewPage. In this case, a page named NewPage will be made.

Here are some design decisions.

The editor allows only paragraph level editing. You can make a line become h1 (header 1), p (paragraph), or pre (program), but you cannot make a word become bold or italic. Link command is only exception case (you can make a word become a link). The reason is web browsers have different behavior on the editor component, and I don't want to support each corner case. Rather than that, decided to provide minimal set of features.

Chalkboard uses IFRAME + editorMode property for the rich text editor. There were another possibility. One is DIV + contentEditable, and another one is FRAMESET. Those were pros and cons. I use IFRAME simply because it was easiest way in Firefox and Safari to make available this layout configuration, but I might change my mind.

My personal history of Web Authorizing Tools (2)

2008-09-02T23:41:00.001-07:00

Tinlizzie Wiki

Tinlizzie Wiki is a wiki written in Tweak. It uses OpenDocument Format (ODF) as data format, and WebDAV as server.

Although data format in StackWiki was Squeak specific binary, In Tinlizzie Wiki existing common format is used. A part of reason why I choose ODF was that it was a research project to find a possibility to exchange eToys content among different platform. So it was necessary to find platform independent and transparent format. ODF, especially its presentation format, was quite close to my demonds which are a) text based b) enable to embed graphics c) enable to use original element d) internal and external link supported.

A ODF file is just a zip archive which includes XML text and multimedia binary files. And it is easy to extract image file in a project by an another tool. Both embeded object and external resource can be represented by common URL notation. And if necessary, new tag for Tweak specific object can be used. For example, a project which includes fully dynamic behavior written as Tweak objects can be viewed on ordinary OpenOffice Org application, although dynamic feature would in it be disabled.

To export Tweak object to ODF as natural as possible, special care was needed to save. It is not the best way to define a new tag for Tweak specific object even though it is possible. It was preferable to map from Tweak to ODF properly. For example, if a Book object in Tweak is stored as a presentation within frame in ODF, the project looks somewhat more normal even on other application.

There is a issue how much detail information is needed to save an object. For example, if a text is saved during its editing, whether if position of the cursor should be saved or not?? There are two strategy in terms of implementation. One is to save everything except specified status (deep copy), another one is to save only specified status. Tinliziie Wiki adopted the latter one although Squeak and Tweak native serialize mechanism were the former.

Saving only specific status has two disadvantages. a) A user might expect to save everything including minor information because combining arbitrary objects in even any peculiar way is possible in Tweak. b) Each new widget needs to implement each exporter. But "saving everything by default" strategy has a problem of compatibility because even just one change of variable name makes trouble for old version. Especially it is problematic for sharing in Internet. So I din't choose this strategy.

WebDAV is used as the server. Both StackWiki and Tinlizzie doesn't need server side logic, but simple storage is required. WebDAV is the best option for that matter. Even version control system can be plugged in the server with Subversion modlule in Apache for free,

Javascript Workspace

Javascript Workspace is a simple web application. It uses bare Javascript on client, and Ruby CGI on server. It behave like a Smalltalk Workspace, and the contents are managed same manner as Wiki.

Let me make sure about workspace again. Workspace is a text editor, and it has two additional commands "do it" and "print it". Do it command envokes a source code selected by user, and print it command output the result into cursor position. The function is similar to REPL shell on dynamic language, but the use case is slightly different. A typical way to use workspace is as an explanation of program. An author writes example source code inside the documentation, so that a user can try actual function while reading a text. Namely, REPL is two ways dialog between a machine and a human, but workspace is tree ways conversation among a machine, an author, and a user.

Workspace is indispensable tool for Smalltak though, which doesn't mean only for Smalltalk. It would be nice if there is a workspace for Javascript language. This was the initial motivation of Javascript Workspace. And then, it was a natural consequence that Wiki was used to save the content because Javascript lives on web browser intrinsically, and there are no way to save to local disk.

During the development, however, I realize that it can be more than just a workspace in terms of media. Javascript workspace has only simple user interface, which includes a couple of buttons and one big text area. Even there are no hyper link nor emphasized text. But variety things can be happend from such minimal configuration by source code. Hyper link is enable to make from location property, rich text can be shown to modify DOM tree, and even game can be made to set up event hander. Source code can do everthing.

Just one textbox on a web page is a very radical idea. This is completely opposite direction to current trend of rich internet application. Web application consists with number of hidden functions these days, but Javascript Workspace can not have any invisible information. Everything what it does is shown to you as source code entirely on the screen. Javascritp Workspace looks like dangerous as it runs any Javascript code, but in fact, it is a quite safe system.

The idea of uset interface of Javascript Workspace is adopted to OMeta/JS.

TileScript

TileScript uses Scriptaculous as GUI library and WebDAV for server storage. JSON is used for its data format.

A TileScript document consists with one or more paragraphs, and a paragraph is either Javascript code, "tile script", or HTML expression. A tile script is set of tiles, which each tile represents some syntactical element in a programming language. A user can connect tiles to construct a program with drag and drop. This is an easy way to make a program avoiding syntax error. Javascript is used to represent more complicated program than tile script. And HTML is used as annotation. It can be seen as rich version of Javascript Workspace.

The initial motivation of TileScript was to remake eToys on the web environment. The research had got started by making tile available on web browser. I considered to use Lively Kernel (SVG), but it was unnecessary if Table element in HTML DOM is used as tile. Scriptaculous is used to keep the source code simple.

After tile is ported, then next step was eToys environment itself which includes event handling, scheduling, and bitmap animation, etc. But those issues seemed too difficult for nature of web document.

Flow layout, which actual position of document elements are dynamically changed by reader's browser environment, is a significant feature of web. An author don't specify concrete position of elements, but rather care about logical structure. And then, a part of document which can not be shown on the screen is accessable by a vertical scrool bar.

On the other hand, eToys provides page layout, which size and position of elements is fixed, and presume particular screen size. Althogh, it is quite fit as a metaphor of physical paper, and best for a graphical environment like eToys, but clumsy operation like zooming and horizontal scrool is required.

Because ultimate goal of TileScript was not just reinventing eToys, but investigating further possibility, flow layout is adopted to TileScript. But still absolute coordination can be supported in form of embeded object even in flow layout. TileScript provides variable watcher like eToys, but those widget is also layouted along with flow.

And then

Now I'm working on next version of Javascript Workspace, which especially its target application is Active Essays. Our group have found that Javascript is quite reasonable tool to show some ideas of computer science. One reason is language's simplicity, and other one is easiness of collaboration. We have a lot of new ideas about programming language, and some of the part should be simple enough to understand even by junior high student. I believe my tool can be used to explain such ideas.

The problem is any project intoroduced here is not intended for real use, rather just for demo or prototype of further real development. So it is not be so useful as it looks because it includes too experimental aspect, too fragile, or too slow. Now I'm thinking that it is not bad idea if I make somewhat stable version of them. Even it might not have exotic feature like tile script, but only basic and simple functions are enough to play with everyone. I really like my first idea of Javascript Workspace, which has only simple text. I admit it is extreme, so next version might support emphasized text and inline image (basic HTML elements) at least.

My personal history of Web Authorizing Tools (1)

2008-09-02T00:10:00.000-07:00

I have been facinated by the idea of combining web authorizing and programming even before I realize it is called Active Essay. Actually I made a numbers of projects along with the idea. Here is a short story of my side.

Scamper Workspace

Scamper Workspace is a extention for Scamper, a web browser written in Squeak. It runs a Smalltalk code on any web page with simple operation.

I often write a tiny source code in Squeak on my blog. And it is natural that you want to run the code without any effort. Usually, you might have to copy and paste from web browser to Squeak. However, although Squeak has already a web browser named "Scamper", why do we need such clumsy way? What's wrong if a web page is just a text as same as other Squeak's text object, and if you can use full features which Squeak IDE has on it. Scamper has very limited as a web browser, but it is just O.K. to examine a code on blog. Only missing thing is a direct way to execute code on web page.

"No Application" is Squeak's original motto. Squeak consists with a number of objects that they have each different tiny functions, and those are connected naturally. In that sense, there are no need of "Application" because application is just an artificial boundary. So it seems to me natural to add Workspace menu into Scamper.

Because all necessary code is already available in Squeak image. The implementation was very easy. However the result was surprisingly attractive and profound. Thoru Yamamoto made a lot of contents for Scamper Workspace and each of them are so exciting.

Typical page written for Scamper Workspace consists with a short story and a couple of codes. A reader would executes the codes while reading a story. This format is very effective when the story is to explain how to make a graphical program in Squeak. Especially the fearure is emphasized when a page consists only text and source code. Even it lookes if it were 1995's website, it make large variety of effects if you run the code. I would say that it could be another possible evolution path for the Web.

StackWiki

StackWiki is a multimedia authorizing tool written in Squeak. Document is saved as original binary format with Swiki Server.

I made StackWiki inspired by Zest and Marmalade. Zest is WYSIWYG authoring tool like Wiki uses local disk, which allows you to make link to other page, and dynamic content in Smalltalk language. Above all, StackWiki emphasis more features in HyperCard and Wiki.

A StackWiki project consists with one or more pages. In WWW, nomally relationship between pages is maintained only hyper link, but there is no structure like hierarchy or order. Some websites attempt to show such structure with navigation link.

Structure among pages helps user to know a point within the context though, it often makes UI complicated. Addition to historical navigation, forward and backword navigation is needed if pages have an order. Above all, if there is hierarchy, three dimensional navigation would be necessary, In StackWiki, only foward and backword structure is used.

With background, you can put together similar structures in multiple place in one. StackWiki only allows one background, although HyperCard could handle more. Background is implemented as a special "background" page. If you add something onto background page, the element are shown same place behind other elements in all pages.

Here is another interesting trade off. How complicated background is needed? Background can be seen as a special version of macro. Macro is a generic term to represent common structure among document. Macro is very useful to reduce redundancy and to improve maintenance factor. However, it is easy to make too complicated macro which includes macro of macro. In end user's point of view, background can be better than full macro.

Each StackWiki's page is stored as a binary format in Swiki serer. Although, Swiki is itself web application, it is used just as file upload feature. StackWiki doesn't use any web standard except HTTP transport to save data. It doesn't so good attitude in terms of data transparency. But it makes the development so quick that it took only three days to implement it.

Active Essays on the Web

2008-08-25T22:26:00.000-07:00

Active Essays is a concept of an essay mixed with computer program coined by Alan Kay. The goal is to explain mathematical or scientific ideas to kids with some interesting program like a computer simulation.

It is similar as a math book which includes a couple of paragraphs and formulas in a page, so that a reader would follow the author's idea by not only reading a text but also "do it" formulas. In case of Active Essays, computer programs are embedded in a page instead of formulas. The largest advantage of Active Essays to math book is that a program is often more dynamic than a formula. It is really fun to see a small lines of code generating a little surprise on screen. The second advantage would be that you don't have to calculate numbers by yourself, a computer must to do better job than you to fiddle variables.

Because Active Essay is not a definitive genre in computer media, so I'm not sure how many active essays have ever been written so far. But HyperCard might be the first media which affords contents mixed with text and program. And you would find many projects with this style in eToys on the web.

What I want to make in next a couple of months is an authoring tool for Active Essays taking advantage of Internet technology. And final goal is to describe entire system as a Active Essay itself.

Internet had a huge impact on our literacy. A lot of amateurs write texts, take photos, and publish to the web them every day. Even it is common that there are families whose every members have each blog. Besides social aspect, a new medium makes a new style of writing. Because actual appearance of writing is depended on reader's web browser or RSS reader, you don't have a full control of final looking of your text. And it is encouraged to take care about logical stricture instead of physical looking. I'm sure the those phenomena affects way of writing.

However, there is a something what have not been changed yet. Internet contents still tend to mimic traditional media. Even though it is highly depend on computer technology, it lacks the most interesting aspect, programming language as a communication media.

Programming language is the primary tool to talk between man and machine, and often even better way to exchange ideas among humans. If there were enough support tools, it would be natural that we write a blog post in programming language, chat with source code, and even write a novel running as a program. Especially when the topic is science or mathematics, programming language would be the most accurate way to tell an idea. It is irony that you can't play a LOGO program when you read LOGO's article on Wikipedia.

However, e.g. while the article on Logo has some good information and examples, none on them can be run, dynamically changed and tried, etc. To me this is outrageous given that the browser was done some years after Hypercard, longer after the Apple ][, and long long after the prior art of the 60s and 70s. -- Alan Kay

What kind of tool is useful to support the idea? Abundant introductory material for science written in eToys would be a good start to examine. And history of Literate Programming shows another interesting aspect of the topic. We would make a progress from these instances in terms of functionality and style.

As functionality, Don Knuth's original idea could evolve more in late binding language like JavaScript than Pascal or C, because WEB's some clumsy convention like macro is needed only for language's inflexibility. And as style, we can avoid limitation coming from physical paper. We don't have to care about pages nor font size, but let's concentrate its logical structure. Hyper link helps to support navigating in random access instead of page numbers.

This is an outline of current thought. I don't think special fancy feature is needed for first prototype (actually, I have made a lot of prototypes already, and this is not exactly the first). The simplest solution might be best. And when we need more, I would make everything from scratch in "Ian's System" later.

Active Essays: http://www.squeakland.org/whatis/a_essays.html
About LogoWiki: http://www.redhat.com/archives/olpc-software/2006-April/msg00035.html

My next three months goals.

2008-08-17T23:04:00.000-07:00

The main point of my previous three months goals was making a text editor on Ian's stuff http://tinlizzie.org/svn/trunk/bottle/. I admit that it has not been finished in terms of building a really useful text widget. Making a text editor looks simple requirement. But even just a substitute widget for Squeak's ParagraphEditor from scratch is quite complicated because it has to deal with a large number of issues like text layout, scroll, font rendering, etc. To make the goal clearer and smaller, I propose one concrete prototype and one real implementation.

An authoring tool for Active Essays on web browser.

It will be direct successor of TileScript http://tinlizzie.org/jstile/ and YUI-based tool for Active Essays http://tinlizzie.org/ometa-js/alan/essays.html. Those were difficult to use even sometimes for myself because they were too heavy and UI was complicated. The focus of this project is brushing up the UI. I will not add a new function, but it will have enough features to express some of examples in Appendix A of STEP paper and Ted's puzzle.

Basic text emphasis (header, list, in-line image)
Wiki based collaboration mechanism.
"include" mechanism
OMeta/JS can be used through "include"
Supported platform: IE6 and Firefox3

A basic widget component for Active Essays on Ian's stuff.

This is a real stuff based on above prototype. Every necessary widget which needed for text editing including text emphasis and in-line image will be implemented. Wiki-based collaboration part won't be included.

This text widget can be used a part of Ted's DB Jr.

Next Goals of My Hobby

2008-08-17T00:24:00.000-07:00

I need my next three month research goals by next Tuesday's meeting. But it is too hard for me to write a documentation before writing source code. Because I'm a 100% pure programmer, I program therefore I am.

But I have to write something anyway. It might be better to write something fun idea before serious documentation. So my brain will switch to "writing mode" smoothly. So I made up my next three month hobby goals. This is it.

Simple Electronics Device

When I was first interested in electronics was around ten years old. I have tried a couple of electronics projects. However, I had never understood how to decide parameters of component. Sure, resistors and ohm's law are pretty easy, but capacitor, transistor, coil, such components are quite difficult. I hate numbers and I am too lazy to handle complicated equation.

Those knowledge is completely unnecessary as a programmer. But same time, I couldn't tolerate the fact that I was ignorance about electronics. So I got started to learn it again a couple of month ago. First goal is to make a simple device to communicate between a computer and real world. This is one of early attempts.

FPGA Block RAM and Demonstration

I happen to get a source code of tiny CPU in Verilog. This is quite beautiful so I bought a Spartan-3A FPGA board for $200 to test this code. FPGA is very interesting gadget. But it requires different ways of thinking against programmer's view. So I still do nothing for it. One of the problems is I don't understand how to use Block RAM. And I don't have nice idea to demonstrate it. Just running a CPU is too boring. I wish I can run the verilog CPU with fun demo perfectly.

A Compiler in Javascript

It still doesn't go beyond a joke. I have played a tiny source code in x86 assembler with GNU as. It was when I was teen that I played Z80 CPU, and I was surprised the some essence of Z80 machine code is still alive in modern CPUs (I had imagined as if modern CPU might have hundreds and thousands of generic registers, but still we call EAX register as an accumulator!). I can't say as it is good though, it is very appealing to me.

So I think that it would be fun if I will make a compiler, assembler and linker in Javascript on a web browser. It might support only one CPU and one platform like x86 linux, still it should be a good exercises.

I could write those ideas more. But this is it today.

Documentation and Programming Language

2008-08-16T01:16:00.000-07:00

A number of major programming languages have its own documentation system. Those variations themselves interest me very much. I picked up some of them and examined.

There are some interesting points to characterize those systems.

Text base VS Memory base: A text based documentation system is implemented as just a text formatter. But some are integrated in the language itself, so you can access the documentation in run time. Emacs Lisp is a significant example of Memory based documentation. I think Memory based documentation is more useful though, sometimes Text base is handy because it doesn't need to load a program only for the documentation.
Description part and Reference part: A documentation might include a description part, which describes overall goals and functions of a part like a class or a module. In contrast with description part, a reference part describes detail of a particular entity like a function or a variable. A reference part is connected with its source code of definition.

Perl

Perl's documentation is described by POD (Plain Old Documentation). POD is a text based system. It has only description part.

POD is a very simple markup language and it has only enough mechanism to represent traditional UNIX manual style which includes certain sections like NAME, SYNOPSIS, DESCRIPTION, etc.

A POD document is embedded in a Perl program as a comment and the Perl's parser just ignores it. A POD parser just ignores program. POD doesn't use any information from the program source code.

http://search.cpan.org/perldoc?perlpod

Java

JavaDoc is the standard documentation tool for Java. It is a text based and it has both description part and reference part.

JavaDoc parser uses not only comments but also program codes to extract type information and class hierarchy. Even it works if you don't write a comment. Very structured HTML documentation is generated by the tool.

http://java.sun.com/j2se/javadoc/

Emacs Lisp

Emacs Lisp has memory based documentation. And it has only reference part.

Emacs Lisp's documentation is highly integrated with Emacs editor. You can access help document for every functions and variables without external tool.

A documentation is described as a first comment in a function or a variable definition. This comment is parsed by Emacs lisp parser, and shown by help command e.g. M-x describe-function. Help system of Emacs Lisp doesn't have description part, but info manual of Emacs lisp complements overall information.

http://www.gnu.org/software/emacs/emacs-lisp-intro/html_node/defun.html

Python

Python has memory based and text based documentation. And it has description part and reference part.

Basically Python's documentation is same as Lisp's. A documentation is described as a first comment of an entity (module, class, variable, etc...), and parsed by Python parser. You can access the documentation through __doc__ property in run time.

I don't know much about Python. But Python is unique because there seems to be a way to extract documentation by text processing like Java and Perl. I'll learn about it later.

http://www.python.org/dev/peps/pep-0257/

Squeak Smalltalk

Squeak Smalltalk doesn't have documentation system. It is not necessary because the IDE is strong enough. But you can see IDE itself as a memory based documentation system. Class comment is used as a description part, and method comment is used as a reference part. Class browser has mode to hide source code but comment.

What does documentation lose?

2008-08-13T23:12:00.000-07:00

I realized that Knuth emphasized narrative way to express a program documentation instead of formal way in "Literate Programming" paper. The reason why WEB needed macro mechanism which Pascal didn't have was that it was needed to make the program flexible enough so that an author can keep order of original thinking flow on the text. Using WEB, whatever each symbol dependents on others, you can write down in any order which you like.

a program is, likewise, ready to comprehend it by learning its various parts in approximately the order in which it was written. -- LITERATE PROGRAMMING

Such consideration have been lost in current documentatin mechanisms like JavaDoc. Major goal of documentation generator for programming language is to make a comprehensive reference, certainly, literary value would be out of scope in those systems. I expect thet My Active Essays project would be a missing link following the lost feature.

However, before diving too deep, I need to know current art and science of program documentation... to be countinued.

Things in my mind recently

2008-08-10T23:34:00.000-07:00

A couple of people asked me about this blog, so I add a new post finally. Because I don't spent my major time for Squeak recently, I changed the title of it with more general flavor than "Squeak Everyday". But "Workspace" is still a keyword from Smalltalk. What I am doing everyday is still dynamic hacking with / without computer.

I'm quite interested in relationship among programming and documentation. It was 80' when Knuth wrote about literate programming http://www.literateprogramming.com/knuthweb.pdf. In this article, he seems to be too excited about his idea that a program is a literature. How far we have elaborated the idea since then. Yes, from Emacs's documentation to JavaDoc, there are a lot of great works. But I'm not sure those are really literature, in other words, those are fun to read. In fact, although I am really fun of API documentation as a hacker, it could be more entertainment. That's what I am thinking these day.

When I was working on my old work http://metatoys.org/propella/js/workspace.cgi/Home (2007), such idea was in my mind, too. And I'm going to visit it again.

ODECo on OLPC

2007-07-05T15:55:00.000-07:00

ODECo http://languagegame.org:8080/ggame/15 is an easy tool kit for making dynamics simulation based on ODE http://www.ode.org. It was developed for Squeak 3.6 in 2004 though. Now I did small fix to work on OLPC etoys and Squeak3.9 for linux (I've tested it on Fedora 7). How to Play: - Download and extract the image. - Install the plugin. (for OLPC, you need to copy ODEPlugin to /usr/lib/squeak/3.9-11/) - Open the image. And try examples. Download: (image + examples + plugins for linux, windows, mac) http://languagegame.org/pub/ODECo-OLPC-2007-07-05.zip Source Code: http://squeaksource.com/ODE/ODE-Plugin-tak.2.mcz http://squeaksource.com/ODE/ODE-Base-tak.39.mcz Website: http://www.languagegame.org:8080/ggame/15

OpenDocument on Squeak

2007-04-07T07:46:00.000-07:00

The end of last month, a french student asked me about my OpenDocumentFormat module for Squeak. ODF??? Yes, I have been written a ODF exporter based on Diego's stuff for a project. http://www.squeaksource.com/OpenDocument.html But I haven't maintenance for a while because I was busy for another project now. I was feeling somewhat guilty because it should be useful even on normal Squeak image if I put some fix. So actually his bug report was a good opportunity to clean up my code. And this is one. It works with Squeakland, Squeak3.8, and Squeak3.9 (YAXO is needed).

UNDO!

2005-09-08T04:27:00.000-07:00

To tell the truth, I haven't cared about UNDO in my work at all. Although I am really interested in brand new feature and fancy demonstration of computer programming, but I never have been concerned real user's experience. Ah, but the time seems to be coming to do about UNDO.

From the stand point of developer, I'm really taking care of undo. Programmer's undo is revision control systems. I love to use Monticello to manage my private project in Squeak, and Subversion for other stuff. I couldn't do anymore unless such system. So I realize that one interesting issue here is why undo system is implemented in various way?!

In minimal scale, we can use just undo key in your editor. Each application has its own undo policy. But the undo buffer is removed when the editor is shutdown. Revision control helps the situation. Besides, sometimes we use more file oriented backup system. So far, my company did differential backup for whole PCs at office into file server (I made the script in Perl). In any case, the goals are same, we want to back some point in history of our computing activity. Are there any core concept in?

Let's talk about other angle of undo. Sometimes undo is implemented as command pattern. Look, we have already have another commands in our etoys system, a tile, so we can imagine if undo buffer is made in etoys tiles. If so, one can learn how tile script is written looking undo history. This is basically same as macro recording (actually, I have learned emacs lisp to record/read macro scripts so far). To achieve such feature, all operation including painting a form and editing a text should be recorded as commands (as HyperTalk has all commands for menu / editor operation).

If all operation is recorded into undo buffer, we can see the undo buffer is document itself. That is an interesting side-effect. Instead of saving document data, we can save a sequence of command. The benefit of it would have a power of generalization. Documents, undo, playback, etc. are formed same way of commands.

To proceed this concept, you can see two different documents as two snapshot from same revision tree. Because certainly, a document is written from empty document. We could see any document of same application derives one root. Hence, any document of Squeak space is regard as different version of one big revision tree described in commands.

I know the idea is based on too simplification. I didn't care any performance issue neither space nor time. But it is important to start where the simple enough point.

Meeting with Julian

2005-09-06T18:06:00.000-07:00

I had meetings with some NICT's researchers and Julian for these two days. Although I still felt slightly jet lag as I just had backed from L.A. on last Sunday, I was excited by a lot of interesting topic at the meeting. I also showed a demonstration about Chibi-Robo stuff. We were talking about to create 3-D scripting system. I realized this goal will be not easy as I expected. I regret my work should be stopped for a while to do another work, but I hope I will be able to make up more generic version. Most interesting topic came in the last dinner. Julian asked to one researcher to use iChat to corroborative work. Now we have enough infrastructure to use video conference, so it is free to use such tool whenever you need. Actually Julian said he already sets up iChat in his home during the entire day, it isn't necessary to turn off, and he can be chatting to his son whenever he want even he is on other side of the world. The important point is you don't need to disturb by noisy bells to aware enyone. One can look at peer not to be busy now nor not. I imagine the world where such video conference technology becomes just a trivial technique, as if today's HTTP or SMTP protocols, the applications of VOIP will become different than today's one (Croquet?!). Early web is just a thing for static document, but now it becomes more dynamic thing. Julian seems to try such experiment with his iChat.

Implementation of TVML with Croquet

2005-09-04T05:41:00.000-07:00

At last entry, I wrote the brief story of the Croquet project with NICT. But now, I am trying to describe more detail implementation of that project as I have a time on the airplane. In term of implementation, I could not say it was a good work. There were a lot of ad-hoc things, dirty patch, and less reusable. All the mess were from its primary goals. - Novice user can use. - Connect with P2P network more than 7 hosts. - The croquet space is regarded as if real NICT interior. Imagine these are not easy in current Croquet release as it is too early stage. For instance, the most strongest demand in here was we didn't have to show any error messages whenever a child touches anywhere, thus it needs a bunch of hooks in UI-related code. The lack of generalization was why I didn't show neither Squeak image nor source code of the project (I am ready to show you my source if you send me a mail). But I think I should contribute parts of codes for this project with some ways. Anyway, let's talk about points of the implementation. ** Describe animation as TVML script Honestly, the script that I described for the project was actually not TVML itself. Although much portion of the script was due to TVML, but I didn't TVML itself because; - Implementation of full set of TVML needs need effort. - I don't want to carry with new syntax into Smalltalk. Finally, the script I introduced looks like; - character:walk (name="Bob" x=10, y=0, z=10) (TVML notation) - Bob tvBehavior runSpec: #(walk x 10 y 0 z 10 length 1000). (Croquet notation) Basic idea is similar, but its expression is different. Croquet version is just a Smalltalk array, and it doesn't need the key "name" because the subject of the script is described as its receiver (Bob tvBavior references a TVML animation engine). Another significant thing is Croquet version stats the length of animation (by millisecond). The length of the animation was essentially platform dependence in original TVML (you can tune with magical numbers to try and error). But in Croquet, accurate timing was needed in network interaction. ** User interface For using as a direction board, I made the user interface be much simpler to eliminate walk-through motion, pop up menu, and dock menu. In stead of that, I put simpler UI. - Both right button and left button mean selecting menu. - The stand point of user does not move. - The degree of view can be moved by operation a mouse (180 width and 30 height). The main differences from regular Croquet is eliminating walk-through. In this work, User doesn't need to move the stand point. Besides, anyone can not make new objects or new windows. ** Network Network was the hardest work to do. Did you know current Croquet just supports up to 3-4 clients? I realize the fact when 3 days before of the demo, it makes my face pale with fear (I regretted I had been too late binder). The problem I met there was; - Become slow down. - A number of shared message was dropped down. - All host was down after one host was down. - Whenever one host is down, I have to restart all the host for synchronizing their states. I couldn't fix those problem in precisely right way as I had no time, but I managed the issue relatively ad-hoc way. - Each host has just atomic, limited states. - Reducing shared message (Cutting lasers, falling down, etc). - The interval of Heart beat (TeaParty>>runKeepAlive) was made longer. - Don't use TCP to avoid blocking. ** The fact we met from the "Open House" event. - It is hard to use if many user operate in same time. - The trackball has too many buttons. - Menus are too large for using in large display. - The pointer cursor is too small to see where is pointed. - It is hard to know each host is connected each other. - Out of memory happened each an hour.

TVML Crouqet exhibition

2005-08-03T08:53:00.000-07:00

TVML Crouqet exhibition The final demonstration of my work about TVML + Croquet had finished last weekend in NICT "Open House" exhibition. We had up to 900 guests including families of the NICT reserchers in the exhibition one day. It was a very exciting day even I was lack of sleep... My work was a kind of computer guidepost. There were six connected Croquet client in varied places like the cafe, gallery, or so, and users can ask the computer the directions for a bunch of display items. In the screen, the space is build as same as actual place, and there is a robotic character named "Chibi-Robo", who leads user to right place. There are a couple of differences than usual Croquet application, most different thing is users never walk in the Croquet space. For example, a Croquet client placed at the cafe is always shows the cafe. If a user moves the mouse, the viewpoint is just rotated, but not moved. If a user wants to follow walking "Chibi-Robo", the user must walk to next Croquet client in actual space. When the user reaches next Croquet, wow! "Chibi-Robo" would be waiting for you in the host. There are six Crouqet, but there is only one "Chibi-Robo" in the space, so children (there are a lot of children!) have to run in the floor to chase Chibi-Robo. Honestly, a number of children played with Chibi-Robo, and began to like Croquet. I thought this was a succesful demonstration. In the technical degree, there were a couple of challenges. Especially, TVML animation and connected 6 hosts are very difficult thing. Actually, I had to reboot each 10-15 minutes for memory leak (about one hour per a host). That was tough experiment for Croquet.

Squeak and stdin/stdout

2005-06-18T09:23:00.000-07:00

I had a discussion with a couple of squeakers today. The main theme of the meeting was about our new web site for Japanese squeak develpers, but there was another interesting topic in the discussion. Standard input and output support in Squeak should be one of key aspect to make Squeak more populer. Sometimes, I've heard that the difficulty to combine another program is a reason that ordinary programmer feels Squeak unfamilier. Although, most of end users don't use stdio/stdin to connect applications these days. But programing fan including me still like to use such traditional way as its simplisity. For example, I am writing this topic with an emacs derivative in WindowsXP. Frequency, I strike a key Control Alt $ to invoke spell checker, or Control C + w to invoke English Japanese dictionary. These function are not in emacs, but external commands with a pipe stream. That was originaly a common operation style in Unix, and we can share bounty of the philosophy even if we use Windows. So non-support of stdio in Squeak seems a defect for hackers daily use. Fortunately, I found a work about this area named OSProcess. That was a impressive work! but a bad news is OSProcess does not support stdio in Windows. I don't know about how OSProcess works, but I start to think to make Windows support of it. Abe-san said he had used a proprietary Smalltalk as a batch language like /bin/sh. I imagine Squeak also should be good language as such glue.

Planning a demonstration of TVML + Croquet

2005-06-14T00:18:00.000-07:00

I have a plan to show my work of TVML + Croquet at NICT public exhibition day the end of July. I have made a PowerPoint slide (I prefer Sqeueak! but they demanded it) and show at last NICT meeting. In the exhibition, main audience are neighborhood children. A lot of kids who are science fun would come. So we have a plan to show 10 PCs connected with Croquet for one whole day. 10 PCs ?! I have never seen such number of hosts connected with Croquet. I know current Croquet network implementation is a bit fragile. Actually, a friend of mine have tried to know how many number computers which can connected in Croquet same time, in that time (2 years ago), the result was only 4 hosts for a few minutes. Besides, this is not just a presentation, but a real demo. the users are hackish children who like to type the keyboard randomly. But I think this is exciting challenge.

Kansai Daigaku Workshop

2005-06-13T18:18:00.000-07:00

Kansai Daigaku Workshop Last week, I conducted a Squeak Etoys workshop at Kansai University. There were 60 students from media course, conputer science, and education cource. For the workshop, I used a special version of Squeak with numeric color control tiles, customized viewer, and morph sharing tab (Super Parts Bin). Although we had troubles some bugs of etoys, I think workshop itself was succees. The workshop was 7 hours. First 3 hours are tutorial for basic technique as "Drive A Car" curriculum. Rest 4 hours are for discussion and practice for theme. The theme of the workshop was "Introduce yourself". Some students could make really impressive work even if such short time. But again, I always think that programming etoys is not easy if you want to complicated project especially for conputer science students. They have complained me Squeak is harder than C++! ;)

Palo Alto a week and tweak

2005-05-10T14:53:00.000-07:00

Today is a day that I have spent at Palo Alto for a week. It is good weather and good temperature. I became fond of bay area as soon as I came here. Now I am sitting front of a desk in HP lab. The laboratory has huge place and a number of numerous scientists from many countries, the accents of discussion through the wall have a exotic taste. I am starting Tweak in here, this is a brand new programming environment that I am puzzled at first. But I begin to like Tweak especially the structure of visual component. The model for solving dependency and constraint is not complete, but neat. Besides, Tweak's small process model is interesting. I don't understand much for Tweak yet, so I still finding my goal of Tweak. During contributing some code for bug fix, I am amusing new world new. My first favorite object is CFieldReference. This behaves as a kind of watcher or proxy object. In Tweak etoys, this object plays interesting role. Making CFieldReference is easy, you would just say; CObject fieldNamed: #fiedName And then, this object watches the value of the field described as #fieldName. If you set the CFieldReference as a player of any player like a drop down list, the list points the value continually. The point what I'm interested in is all fields of any object could be etoy's slot with CFieldReference. In former Squeak etoys world, etoys is in a slight special position. Smalltalk world and etoys world are completely different logic. But in Tweak, all object could have etoys-sh interface. I know having all slot for all variable is not good idea. The system would becomes too complicated withoug proper information hiding manner. But the possibility of integration between etoys world and source code world excites me.

Rendering 3-D objects through dll

2005-03-30T07:57:00.000-08:00

Rendering 3-D objects through dll For a few weeks, I was trying to connect original TVML viewer (written in C) and Croquet through FFI. Once I have written a code for rendering TVML object and playing simple animation in pure Smalltalk, but I took a strategy to make a hybrid system using C and Smalltak because writing all code in Smalltalk would need much time. Connecting original TVML viwer was not so easy, TVML viwer has a bunch of I/O codes, windows thread codes, and infinity loops. I trimed those issue and finally made a DLL wrapper around TVML. It could render TVML animation in Croquet space. Although using the DLL has a couple of problems. I can't see the animation through croquet network, an avator can't touch 3-D objects in TVML. These objects are like ghosts in Croquet world, we can see the objects, but these are not in a memory in Croquet. But it was still fun at least as a demonstration.

OBJ importer

2005-03-08T23:20:00.000-08:00

OBJ importer Finally, I have finished an OBJ importer with PIC image texture. In the fact, the importer lacks some important features like smoothing and normals handling. But I can get a brief snap shot to find out next goal of TVML. Kidawara-san required me special functions on TVML, kinds of network support and interactive user interface, which is not in original TVML. So I have to pursuit how to implement such feature. Anyway, today's result is on Squeakmap; http://map1.squeakfoundation.org/sm/package/0d9b7a69-b789-4490-8c59-24fa32a67af7 (I post the announce to Croquet-list but it didn't delivered yet).

Showing TVML Object and OBJ in Croquet

2005-03-03T17:55:00.000-08:00

TVML Stage In TVML, OBJ file format is used for importing 3-D model. It was terrible that there are a lot of incompatible 3-D format in the world. But anyway, for playback TVML contents in Croquet, I have to make OBJ model exporter. Actually, OBJ format was not complicated. I would say it is relatively straightforward and easier than ASE format (ASE in 3dsmax was ugly), thus parsing the format and building objects were not hard job. But what I remain is texture. In OBJ, PIC file format is used to represent image file. OBJ??? I didn't know such format. In the fact, my picture viewer in windowsXP could not show the PIC image. So today's my task is learning what is PIC and building PIC file importer for Squeak.

Hakodate Workshop

2005-03-02T06:25:00.000-08:00

Hakodate Workshop I and Abe-san conducted Squeak workshop last week in Hakodate Future University. The main part of the workshop was using World-Stethoscope; it is a small device to get external input like luminance, temperature, and voltage. Students made some devices with World-Stethoscope and simple materials like paper cups or balloons. One student developped a kind of toy that is a computer interface for babies. When a user shaked the device, the luminance sensor reacted its movement, and mon's face became glad in a computer screen. The goal of the workshop was discussing possibility of user interface. To invent original user interface, each student exchanged deep ideas of different possibilities of computer devices. That is to say, our workshop was to imagine the feeling of former computer scientists. Although I regret I couldn't tell how the works by students were exciting, that was good experience for me and Abe-san (this was the first World-Stethoscope workshop for university students). After the workshop, we have a plan to publish a booklet with the result of the workshop. It is not only for the record of the workshop, it will includes a text for advanced etoy's user.

TVML

2005-02-08T17:58:00.000-08:00

Finally, I started to make interface between Croquet and TVML. I had a meeting to make sure the role of its implementation with Hayashi-san and Aikawa-san, and they agreed to share current source code written in C. Although I think I should rewrite all aspect of animation in Smalltalk for good compatibility with Croquet, but now I strongly curious how to realize the animation of TVML style. So at first, I am going to make bridge for TVML library. Actually, the bridge of Croquet-to-TVML was quite easy. TVML player already has its API for other windows application, and I could implement a plugin to send TVML text in Croquet to TVML. But I guess getting animation informations form TVML would be harder because TVML animation engine did not suppose to use without TVML native render (I have to use a render in Croquet not TVML Player).

Two Teapot in One Squeak

2005-01-14T02:05:00.000-08:00

I posted a change set at mantis http://bugs.impara.de/view.php?id=831 for running two Teapot in one image. Sometimes I am asked why ODE doesn't work well through network, but I didn't investigate deeply because it is a pain to set up two PC for debugging network. Also, I tried to run two images in one PC, but it doesn't work well. If one image is running, another would be stop. So I decided to run two croquet in one image.

Super Parts Bin

2005-01-05T04:51:00.000-08:00

I made a special parts bin named "Super Parts bin". You can get morphs made by other guys from network. It is named after "Super Swiki", which is a mechanism to share project through network. The reason I made it is because I feel a project is too large to share individual's idea each other. If you could share a morph freely, You can make a collaborative project more easily. In this image, http://metatoys.org/tmp/2005-01-05.zip "LanguageGame" flap tab in right hand is connected my home server. So you can share any morph to drag-n-drop. The image is based on Japanese, but you can change language to push a button of earth in an orange flap.

Japanese Font

2004-12-23T18:14:00.000-08:00

I have almost fixed problems related with Japanese font in Squeak 3.8. These are very complicated, but what is good for me is I could be familiar with font mechanism. The main issue around showing font is how characters should be shown in any rectangle beautifully. Before I fixed, Japanese characters were shown roughly and sometimes tearing. This is because Japanese font is shown implicitly when Japanese character is appeared even if a user select Latin font like accuny. I am interested in Font system because Font in Smalltalk has long history. Abe-san said Smalltalk-80 had StrikeFont class same as Squeak. Also, font is very important as one of the design elements. Personally, I think Squeak should handle commercial fonts more easily. Some squeakers says it is enough to use free fonts. But friends of mime who are artists prefer sophisticated commercial fonts, and I want Squeak attracts such art guys. Especially, Japanese free fonts are much few as there has huge amount of characters. Other aspect why I tried around font is I want to reduce my work. Now Japanese version of Squeak is a bit different with latest English distribution. So I have to port my application for Japanese Squeak when I develop some tools in English. That was much boring thing (actually, sometimes I show English version Squeak demos even for Japanese people, but my mother couldn't understand it and complain me...). When my work finished, and basic parts of both Japanese and English Squeak become same, what I need is just prepare a translation data.

Textual representation of tile scripting

2004-12-07T08:34:00.000-08:00

I couldn't this blog these days because this site was toooo slowly... I am working a lot about etoys these days. Actually, I did not know about etoys well before I had my own etoys workshop because I felt it's ability was too restricted. But I have found that a number of interesting things are remained in the tile scripting system yet. Surely, etoys lacks fancy features as programming language. But I don't think rich system is better system. So I have tried to small improvement etoys without big change. One of my challenges is textual representation of tile scripting. As you know, you can see tile script both as tile and text code (Smalltalk expression). But when you modify the text code, the script would be never seen as tile as the script loses the modified information. I think this is a reason why serious Squeak programmer is not interested in etoys. If tile script is made from textual representation. A programmer could write applications with etoys tile script by Smalltalk code (change set, Monticello, or so). So I have write converter between text and tiles. At first, Smalltalk expression is hard to convert to tiles. And I use S-expression like notation. It works almost well. I am going to make some examples of etoys animation without any drag-n-drop!

ODE Again

2004-11-24T17:05:00.000-08:00

Now I am updating ODE bridge for Croquet. Bernd send me great ODE demo few month ago for impara Croquet. But I have let it alone up to now. There were much interesting gadgets in Bernd's demo, so I was trying to adapt it for Jasmine. Actually this was not difficult and I am going to upload it soon.

LanguageEditor

2004-11-16T07:05:00.000-08:00

I am hacking LangageEditor for few days. LanguageEditor and Babel system is one of major feature of new Squeakland image. You can use multiple language in same Squeak. We have to change words of vocabulary for user interface depend on language, the dictionary of Babel is used for it. My goal is to make it easier editing dictionary. Sometimes, a good engineer is not a good writer. So we need a way of collaborating for making dictionary by both user and programmer. Actually current LanguageEditor is not useful because its output file is internal binary format. I think we should able to exchange, make diff, and be under version control. So I decided to make the file as text. Today I could almost finish the task. I use basic file out (chunk) format for dictionary. I'm going to post the fix when I finish the test in Japanese.

English first / Japanese first

2004-11-09T06:12:00.000-08:00

Now I am writing a paper for C5 conference at next year. My theme is about last workshop and its concept, prospect, and impact of development. At first, I was thinking that I wrote about Croquet, but I couldn't do because the part of what I know about Croquet was too practical. Probably if I am going to use Croquet for actual workshop next year, I can write some article of it. Anyway, I am writing in Japanese. At first I was trying write in English, but I couldn't as my brain was confused. So I decided writing in Japanese first, then translating in English. I don't know sure the way of the writing is better, but the fact is I have not much time.

ALAN-K

2004-11-07T19:56:00.000-08:00

Yesterday, we have a conference named "ALAN-K Project Workshop" for discussing Squeak's curriculum with teachers and developers in Kyoto. Actually, we didn't have enough time to discuss, but it was good opportunity to see how to teach in primary school to high school. What I was most interested in were works by a couple of high school students. In instance, they were making about human powered air plain simulation, and ecosystem simulation; in the simulations, a user could alter each parameter to see what was happen. Other surprised thing was the authors of these works were pretty girls, not nerds wearing glasses. I felt envy the teachers!

Meta-Toys Workshop Finished

2004-10-30T22:37:00.000-07:00

Eventually, my workshop in Hakodate was finished. It was much valuable experience for me as it is my first time to have a workshop directed myself. The workshop will be a start point for making decision of which goal I should do with Squeak and Croquet. The aim of the workshop is to make a tool just for fun. The major of the students is design; almost students in there had experienced to make digital contents with Photoshop, Illustrator, or Flash. I told them to make not only contents, but also a tool for making dynamic contents, which includes paint tool, music composer, game constructor. Students can regard personal computer as meta-media through this workshop. This is different experience than just use softare. At last, almost 70% students could make some tool in 7 hours, and understood this concept despite of that no students know about Squeak before. By the way, I have uploaded the results of the workshop. Althogh all projects was in Japanese, but probably you can understand aim of the project. http://yuri.sakura.ne.jp/~propella/hakodate/hakodateProjects.zip

Hakodate Workshop

2004-10-26T02:49:00.000-07:00

Hakodate is a northern city in Japan. In this week, I am going to have a Squeak workshop at Future University Hakodate. In the fact, this is the first experience to have my own workshop. For the workshop I have planned curriculum and to prepared materials. I am just a programmer, not a teacher, but I believe this workshop should be very helpful to find out my next goal in Sqeuak an Croquet. I know students are not satisfied in just Drive A Car project. They are smart enought and need more interesting subject, but they are not computer science students, so I had to develop different kinds of subject. Also I will use World Stethoscope Morph by Abe-san, and my ODECo morph. At first I considered we use Skeleton or Croquet, but in this time, I abandoned such daring material to be simple enough.

Considering Next Goal

2004-10-19T09:51:00.000-07:00

Considering Next Goal We discussed about 'meta-toys' which is an idea of curriculum in Squeak workshop. Although I have met much technical issue in Squeak, I have not faced conceptual goal including developers, teachers, and students seriously. Now I have my own workshop in school, so it was good opportunity to think in comprehensive ways. 'meta-toys' is a keyword to show what we have to do. I don't want to make Squeak as PowerPoint or Flash. Squeak would be more meta tools. I had analyzed a lot of interesting projects in Squeak, took some aspects to abstract its nature, and discussed about where is a important point of etoys project.

Start Jasmine

2004-10-17T08:22:00.000-07:00

In these few days, I am porting my work into Jasmine. Although I had to learn new things involved with different idea in Jasmine like 'meta' message, but in general, I feel happy to go Jasmine because developing in Jasmine is very easy as I can use cool debugger of Squeak. Also, most important point is that Jasmine is completely open and we can see current problem of latest image with Mantis bug tracking system http://bugs.impara.de/. Actually, I felt worry about a lot of 'red' rows in Mantis (new bugs). But but tracking system has definitely benefit to see goals clearly.

Etoys Project for Meta

2004-10-13T17:36:00.000-07:00

I didn't Croquet yesterday, but I developed a project for the idea of meta with Abe-san for next workshop in Hokkaido. What's meta? It is one of the most important ideas of cs. To show the idea, I made a project that objects dance along with scenario showed as arrow symbols in a play field. In this project, the morphs of arrow symbols are considered as programming commands. The point is I could implement a simple programming language using etoys. According to Abe-san, etoys lacks still minor ability for making meta. In instance, we can get first object as first submorph (as car), but we can not get rest objects (as cdr). If we could use such whole list operation, etoys would get more flexible power.

Keyboard Control

2004-10-12T17:44:00.000-07:00

Croquet was released! We have some discussions in squeak-ja mailing list, too. A lot of people are interested in this new version. I am going to introduce this news in Kansai Open Source Conference next week. Anyway, my today's work was operation with keyboard. Now we can control a soldier character along cursor keys. To realize this feature, what I concerned was where should I put event handling. In normal, event handling is in user camera and each tframe object at pointer, but in the game, we need special keyboard operation like space bar to jump. So I put some trick into TUserCamera >> keyStroke: then I added ability for control general keyboard handling into TSpace. Current my work was about TeaTime Croquet, but I am going to port for jasmine asap. After I collect the change sets of the work, I am going to post these codes (especially, not game specific codes) at mantis.

Animation combination

2004-10-11T07:52:00.000-07:00

I tried to merge characters animations. Actually, the basic animation importer was not so interesting work, but from now, the work would be fun as I could move characters freely as a game. http://languagegame.no-ip.com/pub/combi-2004-10-11.avi

Drone complete

2004-10-09T09:38:00.000-07:00

I have finished the work of basic animation importer for Chain Reaction. Althogh I need more minor fix, I guess it is enough quality as first step. The main problem of Drone was that it includes scale factor. I had to take care of scale value of parent frame when reading position values in animation. In spite of using global coodinate to show vertex, the positions of animation is shown as local. Besides, it is important that these values are only connected to location and scale, but don't connect rotation.

Drone

2004-10-08T09:51:00.000-07:00

I am working around to import Drone animation. Although it should be easier than soldier, but in the fact, the work is much harder as it has complex tree structure. In soldier animation, object tree is consist of helper and geometry, besides Drone is built of group. It is no problem just in static object, but I guess there are some issues in dynamic animation. (Actually, I couldn't find out why Drone doesn't work well yet).

My Mistake of ASE Animation

2004-10-05T18:34:00.000-07:00

There was one assumption which led wrong result in 3ds max's ASE Animation viewer. It is that rotation and position could be handled same way. In OpenGL, 4x4 matrix is a uniform method to deal with rotation, thus I could use matrix combination to combine each property. but in ASE, those are completely different aspect. In animation, position is showed as difference from the position at start frame, but rotation is showed as difference from last frame. So I should handle these properties independently.

ASE Rotation Animation

2004-10-02T09:24:00.000-07:00

I had found out how 3ds max exports rotation animation in ASE file. In spite of translations are described the coordinate from original position, but rotations are described the angle as difference from last frame. I found that during I made a lot of simple examples of animation. Other trick is that frames are interpolated as spline curve, and each delta are slight different. So it's very hard to find out the parameters are delta. Anyway, I found another documentation about ASE file in http://cs.felk.cvut.cz/~sporkaa/doc/3dsnd/thesis.pdf.

3DS Max Trial

2004-10-01T09:38:00.000-07:00

As I couldn't understand the meaning of ASE file, I installed trial version of 3DS Max again. Actually I have installed it before, and I had to reinstall Windows. It was a good opportunity to clean up because my hard drive is almost full by junk. Perhaps if I have another chance dealing with the 3dsmax, it would be better to have the license, but now what I need is just to know how is the basic concept of 3dsmax and how the inner structure is organized as ascii file. Anyway, to make very very simple example again, I found my misunderstanding about the representation of mesh translation soon. But difficult point is rotation. Showing rotation is something different against quaternion, or simple combination of axis and angle. This work is like cryptanalysis.

Animated Mesh

2004-09-28T17:56:00.000-07:00

I am building an animation class for Croquet. I have implemented similar mechanism for 3DSMax ASE format, but I am rebuild new class because old one was depend on ASE. The new one would be more compact, clean and readable. There is an issue about where should an animation scheduler belong. Defining animation is easy because animation could regard as function for time, but scheduling animation has environment specific problems. I have several option. - In a TFrame : TFrame and Morph has basic animation mechanism named stepping. This is most straightforward and easy option. An object starts heartbeat itself. - In global scheduler : In instance, AbstractSound uses global scheduler in Squeak. This way has a benefit that it is easy to blend two motions. I could make a scheduler in each TeaParty in Croquet. - In the user camera : Probably it is good for performance. When we don't see an animated object, the animation doesn't need to move. And to put scheduler in the camera, the frame rate of animation could be changed dynamically according to machine performance. At first, I will use TFrame to set scheduler.

veryDeepCopy

2004-09-27T09:01:00.000-07:00

To implement benchmark, I used #veryDeepCopy to duplicate a mesh object. I didn't sure this message works well in croquet world, but it worked with very small fix. Actually, I just only add TObjectFramework >> veryDeepInner: In general, we have to use deep copy carefully for fear of copying unnecessarily objects. Hence, I should test more. But it was enough to use benchmark scripts.

Copy

2004-09-25T09:18:00.000-07:00

Next month, we have Open-source Conference in Osaka. So we Osaka squeakers proposed to have a booth to show current Squeak activity. In last conference at Tokyo, we had lots of audience in Squeak BOF, I expect to meet many people who are interested in Squeak next month. Anyway, today I was making copies of a soldier animation to get benchmark. Our game would handle up to 6 players, and we have to move these characters smoothly. But I got a trouble to copy an animation object. To copy an animation, we need deep copy. But in general, deep copy has some issue, in instance, a geometry root and an animation frame is pointing same object through different path - diamond reference. In Morphic world, we can use veryDeepCopy to duplicate object. But in Croquet world, the message doesn't work. I don't sure I have to implement deep copier in Croquet world. But just the fact is I couldn't my quota today.

Croquet Unit Test

2004-09-24T09:31:00.000-07:00

Before optimizing 3D animation, I have made small framework for unit test. The framework is to evaluate small croquet code from standard TestRunner in Squeak as unit test. This is just transitional scheme until Croquet's own development environment is finished, but now, it is useful for me. Actually, I had finished this system last month. But it didn't work in latest image because I just use #inWorld:sendTo: to pass a code from Squeak process into croquet world. Sometimes, it didn't work because local TeaParty (Croquet teaParty) has some restriction (joinRestricted) and #inWorld:sendTo: is invoked in this local TeaParty. And I tried to build other TeaParty and evaluate in it each test case. Thus the framework works well. But I have not understood how joinRestricted works yet.

I love Workspace

2004-09-23T08:20:00.000-07:00

When I test a program, usually I use a workspace. In stead of move my mouse, I prefer writing small code which works same way as mouse operation. It is because code has repeatability and it is better if we want to do same test again and again. Also, I use my special workspace named Techo that it has multi-panes and is able to save into file easily. But the problem is I can't use workspace in Croquet. Actually, I fixed my private version of workspace to work in Croquet context several months ago. But last improvement of Croquet have made my workspace crash. I guess the reason is for some changes of TeaTime. TeaTime is an interesting feature in Croquet, but I have not got it completely yet. Anyway, the problem would be good opportunity to understand detail of TeaTime. After fixing the problem, I will try to improve the performance of Croquet.

Skeleton

2004-09-22T07:10:00.000-07:00

Skeleton is one of my recent works which allows you to make declarative programing with spreads sheets like user interface. I have not touched for several months but it need more improvement for practical use. I made some plans for Skeleton. - Making smaller the goal. When I start the project, I want to do a number of things. But apparently, it is bad to do many things same times. What I need would be small declarative engine and supporting to write self descriptive rules, not whole scripting system. - Remove dependency with Connectors. The main reason of using Connectors was making funny animation in etoys. But for the aim, ODECo is better way. Skeleton should be more independent module to collaborate other project like Croquet. - Abandon original scheduling. Skeleton has an original scheduling mechanism for drawing beautiful graph to avoid randomness of step list. But sometimes it occurs confusing problems. - Discard original window. Skeleton has different UI against Squeak standard system windows. I like this design but it is not so useful. I guess it is better to use standard system window to show spread sheet.

Comment

2004-09-22T06:41:00.000-07:00

I noticed this blog didn't allow posting from anonymous. As that is not my intent, I changed the setting. This service has lots of features so I can't take all.

Followed Croquet Development

2004-09-21T07:23:00.000-07:00

I have developed for animation characters, but to tell the truth, my code have not worked in latest Croquet images. Although I had written codes on older images because I prefer stable image as possible, I feel I have to follow current update. So I have worked to merge my code into newest image today. Actually, this was not so easy. Most difficult part is #croquetTwin message which is added last month. The message works like #clone, but sometimes it occurs strange error. I noticed the problem is brought by drop event. When we send a message with any event into croquet world, we had to remove the reference of hand morph (probably any morph). Another reason of difficulty of the problem is that the debugger doesn't work well in the version. I spent all the day to fix the problem. But what I needed was to modify only one line!

Using Player reference in Workspace

2004-09-19T07:07:00.000-07:00

I have tried to hack if we could use a textual reference of a Player in a workspace as substitute for a Morph. It is useful when we introduce Smaltalk language from etoy script editor. But I failed for some reason. Actually we could use a reference name of Player in Japanese, but couldn't in English. That's a bit tricky but the reason is player's reference name is capitalized and it is recogonized as a global variables before parse by the workspace. Japanese has no case sensitive, so when workspace read unknown variable, it is handled by bindingOf: mechanism, but not in English. I gave up the task today. -- The goal is achieved to set Dot (default player) into doItContext. I don't sure the hack is right, but it seems to work well. I posted the changeset to Mantis at http://bugs.impara.de/ that is cool system.

Meta project

2004-09-19T05:24:00.000-07:00

The most interesting issue in Squeak workshop is how we introduce the idea of meta-media. Meta-media is one of the characters of Squeak. We can make a meta poject which aims at make other projcet. Abe-san have made such etoy projcets like a project for making music, a project for paint editor, and tile scripter made from etoys itself. We have to develop a curriculum to bring meta concept for university student.

First meeting

2004-09-18T09:47:00.000-07:00

I have the first meeting about next schedule with Abe-san tonight. Actually, I don't like such meeting but I know I am too lazy without particular deadline. Also we have discussed next image of Japanese Squeak. He said current LanguageEditor is not useful when people translate other words all at once. I don't have exactly idea but that seems an interesting issue.

Animation todo

2004-09-18T04:57:00.000-07:00

Goal 1: Moving all characters

- Drone doesn't work now because probably it has other hierarchical structure.
- Positions of weapon and hip are wrong.
- Follow newest image (current changeset worked two month before image).
- Optimization. Now Each frame and each mesh move independently.

Goal 2: Combining animation, and play with keyboard.

- Animation class it is not depend on ASE format.
- A number of animation should belong to one mesh.
- Keyboard sensor.

Goal3 3: Combining cubes and character.

- Put cubes along to designed data.
- Use ODE to control character.

Karuta

2004-09-17T10:07:00.000-07:00

I learned about history of karuta today. Karuta is a kind of playing card whose root is from tarot in medieval times. I think we will use karuta as material for Squeak workshop next month. The aim of the workshop is "Art and Computer". So my idea is by connecting old media (karuta) and new media (Squeak), we can make new story in computer.

Rockey's boots

2004-09-16T20:54:00.000-07:00

Checking Rockey's Boots - http://www.warrenrobinett.com/rockysboots/ Cool! I noticed it is easyer to make digital circuit simulation than analog in computer.

ODECo

2004-09-15T21:02:00.000-07:00

Some people help me to improve ODECo.

sumim-san's spring demo http://squab.no-ip.com:8080/collab/odespring
Bernd's new function of ODE and 3-D demo.

I think ODE is truly useful tool for Squeak. And I am glad to know people are interested in dynamics simulation on Squeak.

Theme

2004-09-10T02:23:00.000-07:00

This is a small memorandum of a Squeak project.

TAKASHI'S WORKSPACE

Various examples in Haskell's FRP.Reactive

Flapjax vs Tangle

Simple Calorie Calculator in Tangle

Simple Calorie Calculator in Flapjax

Implement Adjustable Number Widget in Flapjax

References

Yet Another "Alligator Eggs!" Animation

Alligator and an egg : λx.x

An alligator eats an egg : (λx.x) y

Take first : (λx.λy. x) a b

Old alligator : (λx.x) ((λy.y) (λz.z))

Color rule : (λx.λy.x) (λy.y)

Omega (Mockingbird hears the Mockingbird song) : (λx.x x) (λx.x x)

Y combinator : λg.(λx.g (x x)) (λx.g (x x))

3 + 4 : (λa.λb.λs.λz.(a s (b s z))) (λs.λz.(s (s (s z)))) (λs.λz.(s (s (s (s z)))))

A hidden story behind the EToys Castle

Tamacola (5)

Prepare your Tamacola environment

Tamacola command

Match against a string constant

Number parser

S-expression parser

Lisp Compiler

Put it in an envelope

Left recursion

Tamacola (4)

Tamacola in Tamacola

Bootstrapping Macros

Eval

Tamacola (3)

How a lisp program is compiled to Tamarin VM

Tamacola (2)

COLA

Tamacola (1)

Table of Contents

Intro

How I started the assembler

Recent Tamarin and ABC tools

Download latest Flex SDK

Download and build Tamarin-central

Build abcdump

How to use abcasm

Wooden Half Adder Video

A group exhibition "First Passage"

First Passage

Wooden half adder

An Assembler for AVM2 using S-Expression

Overview

Background

Usage

Command line tool

Function

Data Type

Syntax

ASM-form

ns_set-section

method-section

signature-subsection

code-subsection

metadata-section

instance-section

class-section

script-section

trait_info

Trait_Slot

Trait_Class

metadata_info

Current Status

Example

References

A simple boolean logic table generator in Javascript

Lazy list and Data flow in Squeak

Introduction

Natural numbers

Define a new LazyList

Iteration, Data flow, Lazy list, and Polymorphic functions

Simulations

A complete example of generating a sound in Flash with SampleDataEvent

A Prolog In Haskell