Learning Erlang and Adobe Flash format same time

Since Luke Gorrie encouraged me to learn Erlang, I have been playing with a bit with the language. One of the most attracting points is bit syntax. When you want a byte array with 65, 66, 67. I can write it as <<65,66,67>>. Further more, When you want a bit pattern of 4 in four bits, 1 in four bits and 0x4243 in 16 bits, you may say <<4:4, 1:4, 16#4243:16>>. What a cool!

$ erl
Eshell V5.6.4  (abort with ^G)
1> <<65,66,67>>.  % numbers are printed with ASCII
<<"ABC">>
2> <<4:4, 1:4, 16#4243:16>>.
<<"ABC">>

I was soon tempted to build a meaningful binary data with this syntax. What does suit? Adobe Flash format would be the weirdest and coolest. It's weirdest because of very bits oriented taste by perhaps historical reasons. It's coolest because it's popular. I want to show you how Erlang is powerful by writing complicated SWF format in the sophisticated syntax. Fortunately, the SWF specification is now free and everybody can write a flash editor by themselves.

First of all, I chose scripting style because I'm too lazy to wait for a compiler. And I start a shell bang line.

#!/usr/bin/env escript
%% -*- erlang -*-
%%! debug verbose

I don't understand how it works, but I just copied from the manual. And then, I have to decide what my first flash file looks like. I'm modest enough to satisfy just a salmon colored window. This is a spec file representing my flash.

% an example flash contents.
spec() ->
    {{frame_size, 2000, 3000},
     {frame_rate, 10},
     {frame_count, 10},
     [{set_background_color, {rgb, 250, 100, 100}},
      {show_frame},
      {end_tag}]}.

This is already reflected by the structure of a flash file. A flash file starts with meta data of file size, frame rate, and frame count. And a bunch of tagged data are followed. Tagged data is the key of extensibility of the flash format. You can embed various kinds of media as various type of tags in a file. In Erlang, fixed sized data is represented as {tuples}, and variable sized data is [lists]. I write each element as {key, arg1, arg2, ...} style, and tags are represented as a list. You might understand each tuple's meaning. You can change the color or size if you like.

Rest of the program reads this data and write a SWF file. I'll explain the program in a top down fashion. The simple main function opens a file and write something. Actual data is created by swf/0 (name/arity is Erlang's way to point to a function) as a sequence of binary.

main(_) ->
    Filename = "simple.swf",
    {ok, S} = file:open(Filename, [binary, write]),
    file:write(S, swf(spec())),
    file:close(S).

Now interesting part begins. If you read Chapter 2: SWF Structure Summary in the SWF specification, you may find that the Erlang program is much more concise and easier to understand than long description in the specification (although, I admit that I omitted some details). Especially, a bit syntax part is obvious, "FWS" is three byte characters, 'Version:8/unsigned-little' is a 8 bits integer for version number, and 'FileLength:32/unsigned-little' follows as a 32 bits little endian unsigned integer for file length. 'Rest/binary' looks tricky, but it means that another binary data Rest is embedded here.

% SWF Structure

swf(Spec) ->
    Version = 10,
    Rest = swf_rest(Spec),
    FileLength = size(Rest) + 8,
    <<"FWS",
     Version:8/unsigned-little,
     FileLength:32/unsigned-little,
     Rest/binary>>.

Because I needed to know the file size in advance, I had to split the main part in two. Swf_rest/1 composes every elements in data, and swf/1 mesures its file size and puts to the top. To pass my sample data to swf_rest/1, I use patterns. Thanks to to tuples and patterns, I don't have to remember an order of arguments. It looks like keyword arguments and more generic. Note that words begins with a lower case like 'frame_size' are symbols and words begins with a capital like 'Width' are variables in Erlang, in my case, 2000 is set to Width. In SWF file, a special unit 'twips' is used, 20 twips are a pixel.

A list comprehensions is another useful feature. I only used it as a shorthand of "map" function like [tag(X) || X <- Tags], which means apply tag/1 function to each element in Tags, but it saved many typing.

swf_rest({{frame_size, Width, Height},
   {frame_rate, FrameRate},
   {frame_count, FrameCount},
   Tags}) ->
    FrameSize = rectangle({rectangle, 0, Width, 0, Height}),
    FrameRateData = fixed8dot8(FrameRate),
    TagsData = list_to_binary([tag(X) || X <- Tags]),
    <<FrameSize/binary,
     FrameRateData/binary,
     FrameCount:16/unsigned-little,
     TagsData/binary>>.

Let's take a look at some detail of data type used in SWF. Color is simple, it is just an array of three bytes for Red, Green, and Blue. The description of 16-bit 8.8 fixed-point number in the specification looks very complicated, but it just said that you have to put the integer part last.

% Basic Data Types

rgb({rgb, R, G, B}) -> <<R:8, G:8, B:8>>.

fixed8dot8(N)->
    IntegerPart = trunc(N),
    SmallPart = trunc((N - IntegerPart) / 1 * 256),
    <<SmallPart:8, IntegerPart:8>>.

This is my favorite part. Rectangles are stored as very bit-oriented way. First, you must get a enough bit size to store each element (Nbits). Second, four elements are stored in this bit size. Third, entire bits should be byte-aligned. This kind of tasks are very tricky in other languages, it require a lots of shifts, ands, ors, etc. But Erlang's bit syntax helps me a lot.

I wrote two (actually three) helper functions to make rectangle/1 simpler. nbits_signed/1 calculate enough bit size for a list of signed integers, and padding/1 fills zeros to align byte width. '/bitstring' is same as '/binary', but you can use it for any odd size of bits data while binary works only byte arrays.

rectangle({rectangle, Xmin, Xmax, Ymin, Ymax}) ->
    Nbits = nbits_signed([Xmin, Xmax, Ymin, Ymax]),
    padding(<< Nbits:5,
      Xmin:Nbits/signed-big,
      Xmax:Nbits/signed-big,
      Ymin:Nbits/signed-big,
      Ymax:Nbits/signed-big>>).

nbits_unsigned(XS) -> % Necessary bits size for a list of integer values.
    Max = lists:max([abs(X) || X <- XS]),
    trunc(math:log(Max) / math:log(2)) + 1.
nbits_signed(XS) -> nbits_unsigned(XS) + 1.

padding(Bits) ->
    Padding = 8 - bit_size(Bits) rem 8,
    <<Bits/bitstring, 0:Padding>>.

Tags are the most important data structure in SWF. There are short and long tags. Short one is used for 62 bytes of data or less. Because tag code and length is packed in one place, you can save precious 16 bits space with short tag! Tricky part is here. In short tag, type code and length are stored respectively 10 bits and 6 bits, however, you have to save it as whole 16 bits integer in a little endian.

Big endian
|Type               |Length     |
|9|8|7|6|5|4|3|2|1|0|5|4|3|2|1|0|

Little endian
|Typ|Length     |Type           |
|1|0|5|4|3|2|1|0|9|8|7|6|5|4|3|2|

What a crazy?! (actually, it is not so strange than it looks if you see carefully. Hint: bits runs right to left, bytes runs left to right). Even these funny endianess conversion is handled easily in Erlang because bit syntax can be used in patterns (input).

% Tag Format

record_header_body(Type, Body) -> record_header_body(Type, Body, size(Body)).

record_header_body(Type, Body, Length) when Length < 63 ->
     <<TagCodeAndLength:16/unsigned-big>> = <<Type:10, Length:6>>,
    [<<TagCodeAndLength:16/unsigned-little>>, Body];

record_header_body(Type, Body, Length) ->
    <<TagCodeAndLength:16/unsigned-big>> = <<Type:10, 63:6>>,
    [<<TagCodeAndLength:16/unsigned-little>>,
     <<Length:32/unsigned-little>>,
     Body].

Finally, I'll show you the format of actual contents. Now I have only three tags for End (type=0), ShowFrame(type=1), and SetBackgroundColor(type=9). You might easily add new tags to support more convincing flash contents. Latest tag number is 91 for DefineFont4, see APPENDIX B Reverse index of tag values in the specification.

% Control Tags

tag({end_tag}) -> record_header_body(0, <<>>);
tag({show_frame}) -> record_header_body(1, <<>>);
tag({set_background_color, RGB}) -> record_header_body(9, rgb(RGB)).

I uploaded an entire program at swf_simple.erl Erlang is famous for its concurrent programming model, but I found another aspects like bit syntax and pattern matching are also attractive and useful to real application.

• The SWF Specification: http://www.adobe.com/devnet/swf/pdf/swf_file_format_spec_v10.pdf