Yammer and Scala

What does this mean? :

def ++ [B >: A, That] (that: GenTraversableOnce[B])(implicit bf: CanBuildFrom[List[A], B, That]): That

I have no idea, but I do know that List(1,2) ++ List(3,4,5) yields List(1, 2, 3, 4, 5). No surprise there. According to Yammer, my faith that ++ does indeed concatenate lists leads to "cargo-culting snippets of code as magic talismans of functionality". Apparently, I must scrutinize and analyze each and every last letter of the method signature to avoid falling into the cargo-cult trap.

It looks to me that Yammer have invented problems where none exists. "Understanding" is a vague concept, but if you have good abstractions, you don't need to understand why something works, only how. At least not until you do need to know.

I'm referring to the Yammer moving away from Scala post btw.

Regarding the Scala community, I'll agree that it's immature. There is no consensus of what good Scala code looks like, and there is a lot of academic noise. An organization that is serious about Scala development needs to come up with their own programming guidelines and maybe also their own utility libraries. Failing to do so will result in code styles all over the place, and an impossible maintenance situation. I expect the Scala community to converge towards a common code style over time, but we're a long way off yet. Today, Scala is mostly driven by academia and hobbyists such as myself, but that will change with industry adoption of Scala.

I have never done any serious performance tuning in Scala, but I'm sure the advise on how to improve performance is accurate in the right context. But in most cases, a difference in an order of magnitude or less is irrelevant, and I from what I hear about e.g. Twitter, when performance is critical, Scala has delivered. Maybe Yammer's needs are different, and can't be met by Scala.

At the end of the day, Scala is just another language, suited for some purposes, but not all.

PS.
I wrote about the academic mind set here, here, and here.

Scala Application Programming is not Math

Programming is one of the most difficult branches of applied mathematics; the poorer mathematicians had better remain pure mathematicians.

- Edsger W. Dijkstra

There is this notion that programming is doing math. It 's not true. Or to put it in a more nuanced manner: It is not true in any meaningful way for an application programmer like me.

I realized how difficult it is to speak to non-application programmers from the reactions I got to last weeks "How to not learn Scala fundamentals". We live in different worlds, me and academia. I tried to expand on my arguments in the follow up, but I probably missed the mark.

The thing is, m.map { t => val (s,i) = t; (s, i+1) }, probably is elegant, and maybe even easy to read for someone exposed to abstract functional code all day long. But for someone who uses Scala as tool, not to improve Scala, but to solve real world problems, abstract descriptions are more or less opaque. We can't easily pick up our rusty university ML experience (if we have that at all) and wrap our heads around "elegant" code like the snippet above. And we shouldn't have to.

If Scala is going to be a main stream language, usable by main stream application programmers, we need to acknowledge that most programmers don't ascribe to the maths based approach to programming.

If you learn Scala without understanding how it all works underneath, you will be less efficient, and sometimes a leaky abstraction will pop up and bite you. But that's ok. You'll still be better off than if you stick to Java. And you will understand more in time, and you'll learn to handle surprises better too.

Scala can boost productivity a lot compared to Java, and you don't need to learn all the "fundamentals" first. That's my message.

XXD Hexdump Tip

I wanted to include a png image in Scala source code as a bytearray. Just use xxd from the terminal:

$ xxd -i icon.png
unsigned char icon_png[] = {
 0x89, 0x50, 0x4e, 0x47, 0x0d, 0x0a, 0x1a, 0x0a, 0x00, 0x00, 0x00, 0x0d,
 0x49, 0x48, 0x44, 0x52, 0x00, 0x00, 0x00, 0x10, 0x00, 0x00, 0x00, 0x10,
 0x08, 0x02, 0x00, 0x00, 0x00, 0x90, 0x91, 0x68, 0x36, 0x00, 0x00, 0x00,
 0x01, 0x73, 0x52, 0x47, 0x42, 0x00, 0xae, 0xce, 0x1c, 0xe9, 0x00, 0x00,
 0x00, 0x8d, 0x49, 0x44, 0x41, 0x54, 0x28, 0xcf, 0x9d, 0x92, 0x41, 0x0e,
 0xc3, 0x20, 0x0c, 0x04, 0xc7, 0x56, 0xc5, 0x03, 0xa2, 0x7e, 0xb4, 0x27,
 0xde, 0x90, 0x13, 0xef, 0x85, 0x4a, 0x56, 0x0f, 0x31, 0x14, 0x51, 0x1a,
 0xda, 0xec, 0xc9, 0x02, 0x0f, 0xac, 0x17, 0x24, 0xcb, 0xc6, 0xa1, 0x5d,
 0x79, 0xb0, 0xd4, 0xcd, 0x5b, 0x81, 0x68, 0xb0, 0x66, 0x24, 0xcb, 0x46,
 0x51, 0x80, 0x04, 0xd1, 0xbc, 0xfe, 0x2e, 0xe5, 0x4f, 0x75, 0x40, 0x34,
 0xf7, 0xb6, 0x06, 0x82, 0x11, 0xcc, 0x87, 0x0e, 0xe6, 0x3b, 0xad, 0x18,
 0x87, 0x2e, 0x7a, 0xd5, 0xd2, 0x45, 0x60, 0x57, 0x77, 0x78, 0x28, 0x55,
 0xc3, 0x75, 0x45, 0xf2, 0xf3, 0x3e, 0x32, 0xa9, 0x66, 0x50, 0x94, 0x30,
 0x06, 0x3d, 0x03, 0xda, 0xd0, 0x33, 0x40, 0x27, 0xc7, 0xb7, 0xd0, 0x7a,
 0x87, 0x69, 0x79, 0xc3, 0xd4, 0xe1, 0x4f, 0x40, 0xef, 0xd0, 0x3f, 0xdf,
 0x79, 0x5f, 0x4b, 0xef, 0xfd, 0x5b, 0x4f, 0xf4, 0xf1, 0xac, 0x2f, 0x59,
 0xca, 0x36, 0xcb, 0xec, 0xe8, 0x2f, 0xba, 0x00, 0x00, 0x00, 0x00, 0x49,
 0x45, 0x4e, 0x44, 0xae, 0x42, 0x60, 0x82
};
unsigned int icon_png_len = 211;
$ 

Copy the hex lines into your source:

val favicon:Array[Byte] = Array(...hex values here...)

Done.
People tend to forget that OS X is built upon a sturdy Unix with all the power of bash and Unix commands.

Follow Up to How to Not Learn Scala Fundamentals

I got some feedback on twitter for "How to Not Learn Scala Fundamentals", so I'll try to clarify my position a bit.

• My post is not intended to be a replacement of the original piece ("Learning Scala? Learn the Fundamentals First"), but rather a criticism of the unnecessary complexity used in his example.
• My example is not supposed to be equivalent from a teaching perspective. It is merely a supposition that if you want to add 1 to a field in a container there are easier ways to do so. And by easier, I mean easier to read. As a reader you are left with the impression that this is how it's done in Scala, and it's not.
• I do not advocate not learning the fundamentals first. I do, however, think that trying to cram 5 fundamentals into one line of code is not conductive to learning those 5 fundamentals. Maybe it works person to person, but it does not work in a blog post.
• My example is not supposed to be cleaner than the original line of code, more on that later.

To reiterate my position from the first post: I firmly believe that if you create an unnatural amalgam of many new concepts in the form of a short code snippet, you will confuse and maybe scare would be converts away. I.e., it is detrimental to getting more people using Scala. This is especially true if you claim that the snippet "is a really simple example".

As to the allegation that my code isn't as clean as the original, I can only agree. At least if immutable is cleaner than mutable.

class Cake(var name:String, var egg:Int)
...
for (cake <- cakes) yield { new Cake(cake.name, cake.egg + 1) }

Fixed. But now I would have to explain yield too. That would be my second example, building on the knowledge gained from the first example.

How to Not Learn Scala Fundamentals

Read this and shed some tears for Scala:
http://tataryn.net/2011/10/learning-scala-learn-the-fundamentals-first/
It's all about a "really simple example":

m.map { t => val (s,i) = t; (s, i+1) }

The author picked this example since it "shows off almost everything you need to know in order to get started ". Had I needed to know this to get started with Scala, I would never have started at all. Scary.

I wonder what the example does...

This is a perfect illustration of how many Scala proponents make Scala look hard, something for the elite. Maybe they find it more important to show off their skills than to help others to get started with Scala.

To be fair, the snippet above is a slide from a talk. Maybe a riddle can be a good presentation tool, but as used in the blog post it only makes Scala look cryptic and difficult to learn.

How would I write the code snippet? I don't know, perhaps something like:

for (cake <- cakes) { cake.egg += 1 }

Or if I had to use the cryptic notation from the snippet:

for (t <- m) { t.i += 1 }

But it doesn't do what the example does, you say. Correct, I cheated a bit and introduced a new class instead of resorting to using tuples:

class Cake(var name:String, var egg:Int)

A small price to pay, wouldn't you agree?

Ok, using tuples here is for demonstration purposes only. But demonstrating several new concepts in one go makes for a steep learning curve indeed.

Having said all this, I must be humble (people who know me would say "Well, that's a first.") and acknowledge that my own published code is wanting. Most of my posts containing code, have the code there to support the text rather than attempting to teach Scala. I write blog post code to be runnable. I should pay more attention to making it readable.

The Zombie Programmer

Should I try to learn new stuff? What should I learn? Learning is an investment, and with all the things I can learn, it's difficult to know where to start. What is most important right now?

Fortunately, learning can be fun, so I read about and experiment with random stuff I find interesting. For me, most of the time learning isn't an investment, it's pure entertainment.

And we all know that entertainment is what's slowly turning us into mindless zombies. That's who I am, the zombie programmer. But a fairly knowledgeable zombie...

Nothing New to See, Move along Folks

I stumbled upon this while researching my opkoko CQRS presentation (in Swedish):

"So CQRS is just what E. Yourdon described 2 decades ago but with a new name and new terms. Furthermore, readonly databases for retrieval of data and DML databases for data manipulation have been common for a long long time. Why is this suddenly ‘new’? ..."

(Comment at http://codebetter.com/gregyoung/2010/02/16/cqrs-task-based-uis-event-sourcing-agh/)

Yes, it is new, but new ideas are seldom all new, usually there are older parallels. It is folly to discard new ideas just because someone said something similar 2 decades ago.

Most useful new ideas are just minor variations of old ideas, sometimes it's just that the environment has changed. Maybe CQRS is just a repurposing of Yourdon's ideas, I wouldn't know, but that is irrelevant. Putting it in the context of domain driven design makes CQRS useful, a new tool in the toolbox.

Prussian Generals

(I gave a 10 minute presentation about the agile Prussian army at the last OpKoKo. It was well received, and I considered sharing the slides, but they don't say that much on their own. I converted it to a blog post with the gist of the presentation instead.)

What can agilistas learn from 19th century Prussian generals? Not much, it would seem. But bear with me, all will be revealed.

But let us start at the beginning. The year is 1800, and the Napoleonic wars are in full swing. Napoleon had a pleasant problem, with too many soldiers to command. Reform was necessary, and he came up with the concept of army corps (Corps d'Armée). A corps was a mini army with its own infantry, cavalry, and artillery, and a fairly independent commander. All in all about 20-30,000 men. The general staff was used to help control all corps in the army. A general staff in the modern sense was a product of the French revolution, and Napoleon benefited enormously from its invention.

French army size at Austerlitz and Wagram

As Napoleon demonstrated the superiority of his style of warfare, other nations started to copy his and France's ideas. Prussia was an early adopter of the French ideas, and it was Prussia who would carry the torch after the demise of Napoleon.

Carl von Clausewitz

Carl von Clausewitz served in the Prussian army, and later in the Russian army during the Napoleonic wars. Soon after the wars ended, in 1816, he started a work on the art of war. He never finished, instead dying from cholera in 1831. His widow published "Vom Kriege" ("On War") posthumously in 1832. Exactly how complete On War was has been hotly debated, but regardless it is considered to be the major treatise on the art and philosophy of war in the western world.

Some interesting points and concepts in On War are:

• Friction, things rarely go as planned when tiny problems accumulates. Roads are more difficult than envisioned, the weather is slowing movement, soldiers get drunk, and so on.
• Fog of war, commanders on all levels will have limited intelligence on which to act, and lack knowledge of the overall picture.
• War is the continuation of policy by other means. A phrase that has been misinterpreted as advocating war over politics, but what Clausewitz really means is that politicians must be the ones to decide if and when war is warranted, and what the goal of war is.

The first two points means that a commander must act and react swiftly on the information at hand. It is as necessary to adjust in the face of bad luck (or preparations), as it is to grasp opportunity when it presents itself.

The third point is important in that it means that war takes place in a context and, like it or not, politicians have a huge impact on how war is waged.

Helmuth von Moltke

Clausewitz found one of his most ardent supporters in Helmuth von Moltke. Moltke coined the phrase "No battle plan survives contact ?with the enemy", advocating adaptability at all levels.

Moltke became Chief of the Prussian General Staff in 1857, a position he held for the next 30 years. When his plans to defeat the Danes in the Second Schleswig War was bodged, he intervened and saved the day. Given this success, Moltke was influential enough to get his way in future wars. Prussia won major wars against Austria (1866), and France (1870-1871).

Though a great strategist, his key to success was meticulous preparations. (Moltke would probably consider that an important part of the strategy.)

• The Prussian war academy trained officers for staff service, and since the cadets were encouraged to socialize and befriend each other, a powerful social network was created within the Prussian army.
• Possible army movements was mapped out in detail. Thus commanders knew their logistical options instead of having to discover everything as they went.
  
• War games was used extensively to gain experience in maneuvering troops. (Moltke used blue figures for friendly troops and red for the enemy, colors that are still used today.)
• Commanders, on all levels, were encouraged to continuously assess the situation, and adjust their plans accordingly. To adjust was more important than to follow orders. One could in fact be reprimanded for following orders, if it meant that opportunities were missed.
  

The social network led to ad hoc collaboration on the battlefield. Intelligence and war game experience led to better, more informed decisions. Command culture led to exploiting enemy weaknesses and countering their strengths, dynamically, on the battle field.

During the second half of the 19th century, the Prussians were invincible.


Wait a Minute

Do you recognize :
1. Individuals and interactions over processes and tools
2. Working software over comprehensive documentation
3. Customer collaboration over contract negotiation
4. Responding to change over following a plan

Here is the complete manifesto

How does the Prussian army hold up against these ideals?

1. While the Prussians instituted processes and used the best tools available, they clearly favored individual decisions and collaboration. The deliberate fostering of informal social networks, and emphasis on responsibility and empowerment of individual officers is dead on.
2. Software was pretty uncommon at the time, but if we broaden software to include process and its documentation, I think it's fair to say that Prussia loved comprehensive documentation, but "No battle plan survives contact with the enemy". If process was broken, it was fixed on the spot.
3. I haven't touched on it, but Moltke collaborate with the political side, with good relations with Bismarck and the king, to ensure that the customer, Prussia, get the most from its army.
4. Moltkes war plans all fail, but the wars are won. Commanders on all levels reacts to and handles setbacks but, perhaps more importantly, grasp opportunities when they arise.

So, in my book the late 19th century army is the poster child of agile.

Conclusions

The Prussian example shows that agile thoughts are not new, and we can learn from their experience. In particular:

• Agile works in large projects
• Requires not only freedom in how follow orders, but even the right to break orders if they are deemed to be wrong (e.g. outdated or too costly)
• Skill and experience is required to navigate the fog and deal with friction along the way
• Local responsibilty requires authority and autonomy
• Communication is vital, and informal communication channels are as important as formal ones

References

• Militär ledning, Marco Smedberg (Swedish)
  
• Om kriget, Carl von Clausewitz (Swedish translation)
  
• The lazy project manager, Peter Taylor
• agilemanifesto.org
• www.clausewitz.com
• The role of leadership in software development - Mary Poppendieck (8:00 - 10:40)
• Wikipedia

Prussian Generals - part 5, Conclusions

Conclusions:

The Prussian example shows that agile thoughts are not new, and we can learn from their experience. In particular:

• Agile works in large projects
• Requires not only freedom in how follow orders, but even the right to break orders if they are deemed to be wrong (e.g. outdated or too costly)
• Skill and experience is required to navigate the fog and deal with friction along the way
• Local responsibilty requires authority and autonomy
• Communication is vital, and informal communication channels are as important as formal ones

References:

• Militär ledning, Marco Smedberg (Swedish)
  
• Om kriget, Carl von Clausewitz (Swedish translation)
  
• The lazy project manager, Peter Taylor
• agilemanifesto.org
• www.clausewitz.com
• The role of leadership in software development - Mary Poppendieck (8:00 - 10:40)
• Wikipedia

A Dysfunctional Web Server Component

Have you ever wanted a tiny web server that you can just embed in your projects? Well, here is one anyway. It's not feature complete, in fact it treats all requests as GET, there is no error handling, and it only produces text/html pages (and by lying at that). It's usable but ... dysfunctional.

Looking at the code, you'll realize it's not fully functional ... dysfunctional.

To use the web server, include it in your code and register page handlers for the pages you need. Look at the UsingTheServer object to see how. It's pretty self-explanatory i hope. The pages map is populated with known pages, and when a page request is made, the web server tried to match the request against the pages map. If found, the partial function is executed.

You can use XML directly in Scala code, and this is where the server lies a bit. Instead of using HTML, we use XML that looks enough like HTML to fool the web browser. (OK, the browser won't be fooled, just tired from the lousy HTML quality nowadays...)

object WebServer {
var server: WebServerInstance = null

def main(args: Array[String]) {
  if (args.length == 1) {
    server = new WebServerInstance(args(0).toInt)
  } else {
    server = new WebServerInstance()
  }

  server.pages += ("/index.html" -> ((method: String, path: String, query:String) =>
  
    
    
        <p>Welcome to { path }. I see you are trying to {method}. The answer is {path + query.reverse}</p>
    
    ))
  server.pages += ("/" -> ((method: String, path: String, query:String) =>
    server.pages("/index.html").tupled(method, path, query)))
  server.start()
}

class WebServerInstance(val port: Int=8080) extends Thread {
  import java.net.{Socket, ServerSocket}
  import actors.Actor
  import java.io.BufferedInputStream
  import collection.mutable.HashMap

  val server = new ServerSocket(port, 100)
  val renderPoolSize = 5
 var renderers = List[PageRenderer]()
  for (i <- 1 to renderPoolSize) renderers = new PageRenderer :: renderers
  renderers.foreach(p => p.start())
  var pages = HashMap("default" -> ((method: String, path: String, query:String) =>
  
    
    
        <p>Welcome to { path }. I see you are trying to {method}. Your question is {query}</p>
    
    ))

  override def run() {
    var i = 0
    while (true) {
      try {
        val connection: Socket = server.accept
        renderers(i) ! (i, connection)
        i = (i+1)%renderPoolSize
      } catch {
        case ex: Exception => ex.printStackTrace()
      }
    }
  }

  class PageRenderer extends Actor {
    def act() {
      loop {
        react {
          case (id: Int, connection: Socket) =>
            try {
              val in = new BufferedInputStream(connection.getInputStream)
              var c: Int = 0
              var request = new StringBuilder()
              do {
                c = in.read()
                if (c != -1) request.append(c.toChar)
              } while ( in.available() > 0)
              println(request)
              val method = request.substring(0, request.indexOf(" "))
              val url = request.substring(method.length() + 1, request.indexOf(" ", method.length()+2))
              val path = if (url.contains("?")) url.substring(0, url.indexOf("?")) else url
              val query = if (url.contains("?")) url.substring(url.indexOf("?") + 1) else ""
              val out = connection.getOutputStream
              val html = if (pages.contains(path)) pages(path).tupled(method, path, query)
                else pages("default").tupled(method, path, query)
              val response = new StringBuilder("HTTP/1.0 200 OK\r\n" +
                "Server: labserver\r\n" +
                "Content-length: " + html.toString().length() + "\r\n" +
                "Content-type: text/html\r\n" +
                "\r\n" +
                html
                );
              val binaryResponse: Array[Byte] = response.toArray.map(c => c.toByte)
              out.write(binaryResponse)
              in.close()
              out.close()
              connection.close()
            } catch {
              case ex: Exception => ex.printStackTrace()
            }
        }
      }
    }
  }
}
}

object UsingTheServer {

def main(args: Array[String]) {
  WebServer.main(Array("8080"))
  MyStuff.getClass
}

object MyStuff {
  var events = List[String]()
  WebServer.server.pages += ("/addStuff" -> ((method: String, path: String, query:String) =>
  
    
    
        <form action="/doAddStuff" method="GET">
          <p>Submit your title.</p>
          <input name="title" type="text"/>
          <input type="submit"/>
        </form>
    
    ))
  WebServer.server.pages += ("/doAddStuff" -> ((method: String, path: String, query:String) => {
    events = events ++ (query :: Nil)
    events.foreach(println)
    WebServer.server.pages("/addStuff").tupled(method, path, query)
  }))
}
}

Prussian Generals - part 4, Take a Step Back

Do you recognize :
1. Individuals and interactions over processes and tools
2. Working software over comprehensive documentation
3. Customer collaboration over contract negotiation
4. Responding to change over following a plan

Here's the complete text

How does the Prussian army hold up against these ideals?

1. While the Prussians instituted processes and used the best tools available, they clearly favored individual decisions and collaboration. The deliberate fostering of informal social networks, and emphasis on responsibility and empowerment of individual officers is dead on.
2. Software was pretty uncommon at the time, but if we broaden software to include process and its documentation, I think it's fair to say that Prussia loved comprehensive documentation, but "No battle plan survives contact 
with the enemy". If process was broken, fix it on the spot.
3. I haven't touched on it, but Moltke collaborated with the political side, with good relations with Bismarck and the king, to ensure that the customer, Prussia, got the most from the army.
4. Moltkes war plans all fail, but the wars are won. Commanders on all levels reacts to and handles setbacks but, perhaps more importantly, grasp opportunities when they arise.

So, in my book the late 19th century army is the poster child of agile.

Prussian Generals - part 3, Moltke

Clausewitz found one of his most ardent supporters in Helmuth von Moltke. Moltke coined the phrase "No battle plan survives contact 
with the enemy", advocating adaptability at all levels.

Moltke became Chief of the Prussian General Staff in 1857, a position he held for the next 30 years. When his plans to defeat the Danes in the Second Schleswig War was bodged, he intervened and saved the day. Given this success, Moltke was influential enough to get his way in future wars. Prussia won major wars against Austria (1866), and France (1870-1871).

Though a great strategist, his key to success was meticulous preparations. (Moltke would probably consider that an important part of the strategy.)

• The Prussian war academy trained officers for staff service, and since the cadets were encouraged to socialize and befriend each other, a powerful social network was created within the Prussian army.
• Possible army movements was mapped out in detail. Thus commanders knew their logistical options instead of having to discover everything as they went.
  
• War games was used extensively to gain experience in maneuvering troops. (Moltke used blue figures for friendly troops and red for the enemy, colors that are still used today.)
• Commanders, on all levels, were encouraged to continuously assess the situation, and adjust their plans accordingly. To adjust was more important than to follow orders. One could in fact be reprimanded for following orders, if it meant that opportunities were missed.
  

The social network led to ad hoc collaboration on the battlefield. Intelligence and war game experience led to better, more informed decisions. Command culture led to exploiting enemy weaknesses and countering their strengths, dynamically, on the battle field.

During the second half of the 19th century, the Prussians were invincible.

part 4

Everything in a Single File

You may have noticed that I prefer to write my Scala examples as a single listing, I also only use what is built into Scala, no third party libraries.

The advantage for me is that I don't have to find some other place to host the code, and it saves me the trouble of having to explain how to set things up in order get the code to run. The advantage for you, the reader, is that you can simply copy and paste into a xxx.scala file in your Scala IDE or editor of choice, compile, and expect the code to work (at least if you use the same version of Scala as I do).

The downside is that I may have to roll my own implementations of readily available components, and that the programming style will not lend itself to TDD. For the kind of toy projects I currently do, I think the trade-off is worth it, and creating my own components can be great fun. Not invented here syndrome, here I come :)

Of course, I won't go to extremes with this philosophy, a particular subject may require another approach.

Stay tuned for The Dysfunctional Web Server, my toy web server component that I'll use in future labs.

Prussian Generals - part 2, Carl von Clausewitz

Carl von Clausewitz served in the Prussian army, and later in the Russian army during the Napoleonic wars. Soon after the wars ended, in 1816, he started a work on the art of war. He never finished, instead dying from cholera in 1831. His widow published "Vom Kriege" ("On War") posthumously in 1832. Exactly how complete On War was has been hotly debated, but regardless it is considered to be the major treatise on the art and philosophy of war in the western world.

Some interesting points and concepts in On War are:

• Friction, things rarely go as planned when tiny problems accumulates. Roads are more difficult than envisioned, the weather is slowing movement, soldiers get drunk, and so on.
• Fog of war, commanders on all levels will have limited intelligence on which to act, and lack knowledge of the overall picture.
• War is the continuation of policy by other means. A phrase that has been misinterpreted as advocating war over politics, but what Clausewitz really means is that politicians must be the ones to decide if and when war is warranted, and what the goal of war is.

The first two points means that a commander must act and react swiftly on the information at hand. It is as necessary to adjust in the face of bad luck (or preparations), as it is to grasp opportunity when it presents itself.

The third point is important in that it means that war takes place in a context and, like it or not, politicians have a huge impact on how war is waged.

part 3

Prussian Generals - Overview

I gave a 10 minute presentation about the agile Prussian army at the last OpKoKo. It was well received, and I have considered sharing the slides, but they don't say that much on their own. Therefore I have decided to write a series of short blog posts with the gist of the presentation.

Since the series is complete, I put all parts in one post, go here to read it: Prussian Generals

(I will update this post with links to the various parts as they are published.)

1. Setting the Stage
2. Carl von Clausewitz
3. Helmuth von Moltke
4. Take a Step Back
5. Conclusion

)

Prussian Generals - part 1, setting the stage

Since the series is complete, I put all parts in one post, go here to read it: Prussian Generals

What can agilistas learn from 19th century Prussian generals? Not much, it would seem. But bear with me, all will be revealed.

But let us start at the beginning. The year was 1800, and the Napoleonic wars are in full swing. Napoleon had a pleasant problem, with too many soldiers to command. Reform was necessary, and he came up with the concept of army corps (Corps d'Armée). A corps was a mini army with its own infantry, cavalry, and artillery, and a fairly independent commander. All in all about 20-30,000 men. The general staff was used to help control all corps in the army. A general staff in the modern sense was a product of the French revolution, and Napoleon benefited enormously from its invention.

French army size at Austerlitz and Wagram

As Napoleon demonstrated the superiority of his style of warfare, other nations started to copy his and France' ideas. Prussia was an early adopter of the French ideas, and it was Prussia who would carry the torch after the demise of Napoleon.

part 2

Javaites won't do Scala

David Pollack writes "Scala use is less good than Java use for at least half of all Java projects". And I agree with his conclusion that

"Scala is an inappropriate language for the majority of Java developers and cannot be expected to replace Java because for at least 50% of Java developers, Scala's difficulty outweighs its value."

But it is dangerous to draw the conclusion that Scala isn't a suitable replacement for Java :-). I don't have the experience David has using Scala. I did however make the transition to object oriented programming in the late 1990:s while working at a large company, and I was responsible for Java technology at said company for a brief period of time.

When Java and Microsoft took over the world, it wasn't by replacing Cobol and RPG with Java and VB. Instead, new projects used these technologies, and there were ambitious programmers ready to man those projects. First came the r&d labs, followed by pilots, and years later, full scale projects. Most old school developers didn't learn Java until much later, some still haven't. But as time goes on, more and more old systems are retired, but more importantly, the number of new applications is exploding, swamping the number of old ones. And I don't believe we have seen the end of that trend.

So the future of Scala will not depend on if most Java developers transitions to Scala. It will depend on whether Scala can compete with other newcomers on the corporate scene.

One final thought, writing Java in Scala will not necessarily make projects fail, and is inevitable in a large company. The code will improve over time (with new projects), and the productivity gains realized, but it will take several years.

Scala is so very hard ... not

It is a bit tiresome to read about how hard Scala is to use. If Scala is hard, then so is every other modern language. Yes, even Java, the favorite language to compare Scala to. For old farts like me, the increased complexity has been gradual, but very real non the less.

When I started as a junior programmer back in the eighties, recursion was considered really advanced and difficult to grasp. A few years earlier, Ada was criticized for being too large and complex. A few years ago, Visual Basic came under fire for much the same reason. Progress is controversial.

So is Scala hard and complex? If you intend to use all of Scala, then it certainly is. But few people have the need to use all the nooks and crannies of a programming language. And we see many features only as consumers, not creators. For instance, when coding Java, I use generics a lot, but I rarely create generic methods. It's not needed for business applications.

The days when one could learn everything in a language lie way back in time. No longer can you judge complexity of a language simply by adding together the complexity of all the parts of a language. Judgement must be metered out in light of the intended application.

Scala is very easy to learn enough of to get started. The road from there may take you to easy or hard parts of Scala, but that's no different from other modern languages. If you need to code complex programs, it gets hard. (Of course, some things may be easy in one language, hard in another, but you can't escape essential difficulty.)

If you don't need to write DSL parsers, you can safely skip that part of Scala. If you find yourself in a project where it's needed, and you find it difficult to understand, Scala is the least of your problems.

If you don't write libraries or application servers, you probably don't need to worry too much about monads, currying or partially applied functions.

If you don't need XML... You get the drift.

To summarize, as a general programming language, Scala is easier than most languages. It has some features that could be considered hard, but just like Java, that's parts that's either not used in day to day programming, or that should only be used when the task is inherently difficult. Don't worry, be happy.

I Don't Know

Observation: People who know what they're talking about are likely to say "I don't know" a lot. Confidence in your own ability also means that you have a clear perception of where the boundaries of your knowledge lie.

On the Nokia vs Apple Settlement

The common interpretation is that Nokia won and Apple lost. Even John Gruber of Daring Fireball fame just reported the fact, without any comment whatsoever. However, the settlement is secret, so it's just as easy to argue that Apple won: Nokia wanted Apple to pay a hefty fee to license patents necessary to build iPhone. When Apple refused, Nokia had to back down and now gets a fraction of what they really wanted. I suspect that the truth is somewhere in between.

Remember that when the process started, Apple claimed that Nokia demanded unreasonable license terms. Apple did not say they wouldn't need to license Nokia's patents, but that Nokia refused to offer RAND* terms as required by the standards the patents applied to.

Sure, Apple sued Nokia in turn, but that is standard procedure in this kind of litigation. If you don't, you won't have anything to bargain with down the road.

The fact that they settled as fast as they did indicates that there was no real dispute about principles, and both sides are happy enough with the deal. The law suit was just part of license negotiations, maybe because Apple was in no hurry to get a license agreement in place, and Nokia decided to speed things up.

* Reasonable and non-discriminatory licensing (RAND)

Probability Sucks: Three Cards

A long time ago I encountered a simple problem about probability. For me, it was an eye opener. Common sense did not apply, and I was captivated by how we generalize in erroneous ways.

The problem was this:

You have three cards, one card is black on both sides, one card is white on both sides, and the last card is black on one side and white on one side. If you draw a card that is white on the front, what is the probability that it is whit on the other side?

The answer seems obvious, 50%, right? Or maybe not. Let's write a short program to test the hypothesis. Scala is the natural choice, of course:

object CardSimulation {
import util.Random
def main(args: Array[String]) {
  val stack = Array(new Card(Face.White, Face.White), new Card(Face.White, Face.Black), new Card(Face.Black, Face.Black))
  println("Welcome to the card simulation")
  print("Enter number of times to pick a card> ")
  while (true) {
    try {
      val line = Console.in.readLine
      if (line == "quit") exit()
      var tries: Int = line.toInt
      var results = List[Card]()
      for (x <- 0 until tries) {
        val cardNumber = Random.nextInt(3)
        val facingUp = Random.nextBoolean()
        val card = new Card(stack(cardNumber).front, stack(cardNumber).back, facingUp)
        results = card :: results
        if (card.upSide == Face.White) print("w")
        else print("b")
      }
      println("\n skipped " + results.count(_.upSide == Face.Black) + " black cards")
      val totalWhites = results.count(_.upSide == Face.White)
      println(" using " + totalWhites + " white cards")
      val otherSideWhites = results.filter(_.upSide == Face.White).count(_.otherSide == Face.White)
      println("  of which " + otherSideWhites + " had a white back side, or " + (100D*otherSideWhites/totalWhites) + "%")
      println(" ok")
      print("> ")
    } catch {
      case ex: Exception => println("Enter an integer or type quit to exit")
    }
  }
}

object Face extends Enumeration {
  val White = Value
  val Black = Value
}

class Card(val front: Face.Value, val back: Face.Value, var faceUp:Boolean = true) {
  def upSide: Face.Value = {
    if (faceUp) front
    else back
  }
  def otherSide: Face.Value = {
    if (faceUp) back
    else front
  }
}
}

A sample run looks like this:

Welcome to the card simulation
Enter number of times to pick a card> 55
bwwbbwwwbwwwbwwbbwbbbbbwbbwbbwwwbwbbbwwwbwbbwbwbwwbbwbb
skipped 29 black cards
using 26 white cards
of which 18 had a white back side, or 69.23076923076923%
ok
> 55
wwwbbwbwwwwbwbwwbbbbwbwbwwwbwbbbbwbwwbwbbbbwbwwwbbbbbbb
skipped 30 black cards
using 25 white cards
of which 18 had a white back side, or 72.0%
ok
> 55
wbwwwwwbwbwbbbbwbwwwbbbbbbbbbbbbwbbwbbbwwwwbwwwbwbbwbww
skipped 30 black cards
using 25 white cards
of which 15 had a white back side, or 60.0%
ok
>

Hm, either I'm extremely lucky, or something is a bit fishy... Lets see, with three cards with two sides each we get 6 possible outcomes, 3 outcomes are black and the problem states that we have a white card, so 3 outcomes remains. 3 outcomes, I think we're on to something. The possibilities are (1)white front of card 1, (2)white back of card 1, and (3)white front of card 2.

Notice that (1) and (2) are white on the other side, and only (3) is black on the other side. I.e. the probability that the color of the other side is 2/3. This explains the output of our test program. Not so difficult to understand once you know you need to go a little bit deeper than just going by first impressions.

The take away, at least for me, is that probability is deceptive, and it is easy to assume wrong, even when it's fairly easy to reason your way to a correct answer.

Scala and FORTH - part 3

I upgraded to IntelliJ 10.5 and the new Scala plug-in, and I started getting warnings. I have fixed them, but I won't go into specifics, check the code at the bottom to see the changes.

Ok, numbers, how can they be compiled? In Forth the word "lit" is used behind the scenes to put compiled numbers on the stack, ": 1+ 1 + ;" compiles to "1+ => lit 1 +". In ScalaFORTH we'll use a variation of that concept. We don't explicitly use memory to store word definitions, but Scala List:s, and we can use a "Lit" class to put a number on the stack.

  class Lit(val num :BigDecimal) extends AbstractWord {
    val name:String = "lit " + num
    def eval() {
      stack = num :: stack
    }
  }

Notice that it's a class, not an object, and we use it by instantiating a new instance for every number that we want to compile.

  def main(args: Array[String]) {
    ...
        if (mode == Mode.interpret) {
          ...
        } else {
          if(nameDefinition) {
            ...
          } else {
            dict.get(item) match {
              case Some(s:AbstractWord) => s.compile()
              case None => dict.get("lit " + BigDecimal.apply(item)) match {
                case Some(s:AbstractWord) => s.compile()
                case None =>
                  val num = BigDecimal.apply(item)
                  dict += ("lit " + num -> new Lit(num))
                  compilationStack = compilationStack ::: ("lit " + num) :: Nil
              }
            }
        ...
  }

We use the name "lit " + num, and since it contains a space, there is no way to define a word with that name from within ScalaFORTH. Since we put the created object in the dictionary, we can reuse lit:s every time a specific number is needed.

The vocabulary is somewhat limited, but let's try it out anyway:
Welcome to scalaFORTH

> : 1. 1 . ;
 ok
> 1.
1  ok
>

This was easier done than said so let's clean up the code a bit and build up a basic vocabulary before we call it a day.

If we want to be able to do math without getting ArithmeticException:s all over the place, we need:

  val mc = java.math.MathContext.DECIMAL128

Add a helper class for 2 parameter words that leaves 1 number on the stack:

  abstract class Abstract2to1ParamWord extends AbstractWord {
    var left: BigDecimal = 0
    var right: BigDecimal = 0
    def prepare() {
      right = stack.head
      stack = stack.tail
      left = stack.head
    }
  }

Add some primitive words:

  def initDict() {
    ...
    dict += ("+" -> Plus)
    dict += ("-" -> Minus)
    dict += ("*" -> Times)
    dict += ("/" -> DividedBy)
    dict += ("pick" -> Pick)
    dict += ("nrot" -> Nrot)
    dict += ("minus" -> UnaryMinus)
    dict += ("0=" -> ZeroEquals)
    dict += ("0<" -> ZeroLess)
    dict += ("drop" -> Drop)
  }

You can see their implementations in the complete listing down below. I refactored the main loop and added a simplistic error handling so we don't get shut down every time an error occurs:

  def main(args: Array[String]) {
    initDict()
    interpret(forthVocabulary)
    Console.println("Welcome to scalaFORTH")
    Console.print("> ")
    while (true) {
      try {
        val line = Console.in.readLine
        interpret(line)
        Console.println(" ok")
        Console.print("> ")
      } catch {
        case ex: Exception => ex.printStackTrace()
      }
    }
  }

The line interpreter has been moved to its own method, and now we can initialize ScalaFORTH with regular word definitions at start up:

  val forthVocabulary ="" +
    ": 1+ 1 + ; " +
    ": 1- 1 - ; " +
    ": over 2 pick ; " +
    ": 2over 4 pick 4 pick ; " +
    ": dup 1 pick ; " +
    ": 2dup over over ; " +
    ": 2drop drop drop ; " +
    ": swap 2 nrot ; " +
    ": 2swap 4 nrot 4 nrot ; " +
    ": rot 3 nrot ; " +
    ": 2rot 6 nrot 6 nrot ; " +
    ": = - 0= ; " +
    ": < - 0< ; " +
    ": > swap - 0< ; " +
    ": */ rot rot * swap / ; " +
    ""

Excellent! Now we have an unusable Forth system, but it starts to look like Forth at least:

> : add5 5 + ;
 ok
> 1 3 / add5 dup dup 1+ - .
-1.000000000000000000000000000000000  ok
> .
5.333333333333333333333333333333333  ok

What happened? Define a new word that adds 5 to the top item on the stack, then use the new word together with some built in words: Divide 1 by 3 and leave the result on the stack, add 5 and leave result on stack, duplicate the result twice, add 1 to the top duplicate, subtract the top duplicate from the second item on the stack, print the top of the stack, twice. Maybe it's easier to understand if we track the stack contents (top of stack to the right):
"1" 1
"3" 1 3
"/" .333333333333333333333333333333333
"add5" 5.333333333333333333333333333333333
"dup" 5.3... 5.3...
"dup" 5.3... 5.3... 5.3...
"1+" 5.3... 5.3... 6.3...
"-" 5.3... -1
"." 5.333333333333333333333333333333333
"." empty

Next time we'll implement "if". I think that's going to be a bit tricky, so stay tuned.

object FORTHv5 {
  var stack = List[BigDecimal]()
  var dict = Map[String, AbstractWord]()
  var compilationStack = List[String]()
  var mode = Mode.interpret
  var nameDefinition = true
  val mc = java.math.MathContext.DECIMAL128
  val forthVocabulary ="" +
    ": 1+ 1 + ; " +
    ": 1- 1 - ; " +
    ": over 2 pick ; " +
    ": 2over 4 pick 4 pick ; " +
    ": dup 1 pick ; " +
    ": 2dup over over ; " +
    ": 2drop drop drop ; " +
    ": swap 2 nrot ; " +
    ": 2swap 4 nrot 4 nrot ; " +
    ": rot 3 nrot ; " +
    ": 2rot 6 nrot 6 nrot ; " +
    ": = - 0= ; " +
    ": < - 0< ; " +
    ": > swap - 0< ; " +
    ": */ rot rot * swap / ; " +
    ""

  def main(args: Array[String]) {
    initDict()
    interpret(forthVocabulary)
    Console.println("Welcome to scalaFORTH")
    Console.print("> ")
    while (true) {
      try {
        val line = Console.in.readLine
        interpret(line)
        Console.println(" ok")
        Console.print("> ")
      } catch {
        case ex: Exception => ex.printStackTrace()
      }
    }
  }

  def interpret(input: String) {
    val items = input.split(' ')
    for (item <- items) {
      if (mode == Mode.interpret) {
        dict.get(item) match {
          case Some(s:AbstractWord) => s.eval()
          case None => stack = BigDecimal.apply(item) :: stack
        }
      } else {
        if(nameDefinition) {
          compilationStack = item :: compilationStack
          nameDefinition = false
        } else {
          dict.get(item) match {
            case Some(s:AbstractWord) => s.compile()
            case None => dict.get("lit " + BigDecimal.apply(item)) match {
              case Some(s:AbstractWord) => s.compile()
              case None =>
                val num = BigDecimal.apply(item)
                dict += ("lit " + num -> new Lit(num))
                compilationStack = compilationStack ::: ("lit " + num) :: Nil
            }
          }
        }
      }
    }
  }

  def initDict() {
    dict += ("." -> Dot)
    dict += ("quit" -> Quit)
    dict += (":" -> Colon)
    dict += (";" -> SemiColon)
    dict += ("+" -> Plus)
    dict += ("-" -> Minus)
    dict += ("*" -> Times)
    dict += ("/" -> DividedBy)
    dict += ("pick" -> Pick)
    dict += ("nrot" -> Nrot)
    dict += ("minus" -> UnaryMinus)
    dict += ("0=" -> ZeroEquals)
    dict += ("0<" -> ZeroLess)
    dict += ("drop" -> Drop)
  }

  abstract class AbstractWord {
    val name:String
    def eval()
    def compile() {
      compilationStack = compilationStack ::: name :: Nil
    }
  }

  class Word(var immediate: Boolean = false, val name: String, 
             val words:List[String] = List()) extends AbstractWord {
    def eval() {
      for (word <- words) {
        dict.get(word) match {
          case Some(s:AbstractWord) => s.eval()
          case None => Console.err.println("Word not found: " + word)
        }
      }
    }
  }

  class Lit(val num :BigDecimal) extends AbstractWord {
    val name:String = "lit " + num
    def eval() {
      stack = num :: stack
    }
  }

  object Dot extends AbstractWord {
    val name = "."
    def eval() {
      Console.out.print(stack.head + " ")
      stack = stack.tail
    }
  }

  object Pick extends AbstractWord {
    val name = "pick"
    def eval() {
      stack = stack(stack.head.intValue()) :: stack.tail
    }
  }

  object Drop extends AbstractWord {
    val name = "drop"
    def eval() {
      stack = stack.tail
    }
  }

  object Nrot extends AbstractWord {
    val name = "nrot"
    def eval() {
      val depth = stack.head.intValue()
      val newTop = stack(depth)
      val oldTop = stack.tail.take(depth - 1)
      val oldBottom = stack.drop(depth + 1)
      stack = newTop :: oldTop ::: oldBottom
    }
  }

  object UnaryMinus extends AbstractWord {
    val name = "minus"
    def eval() {
      stack = (- stack.head) :: stack.tail
    }
  }

  object ZeroEquals extends AbstractWord {
    val name = "0="
    def eval() {
      stack = (if (stack.head == 0) -1 else 0) :: stack.tail
    }
  }

  object ZeroLess extends AbstractWord {
    val name = "0<"
    def eval() {
      stack = (if (stack.head < 0) -1 else 0) :: stack.tail
    }
  }

  abstract class Abstract2to1ParamWord extends AbstractWord {
    var left: BigDecimal = 0
    var right: BigDecimal = 0
    def prepare() {
      right = stack.head
      stack = stack.tail
      left = stack.head
    }
  }

  object Plus extends Abstract2to1ParamWord {
    val name = "+"
    def eval() {
      prepare()
      stack = (left + right) :: stack.tail
    }
  }

  object Minus extends Abstract2to1ParamWord {
    val name = "-"
    def eval() {
      prepare()
      stack = (left - right) :: stack.tail
    }
  }

  object Times extends Abstract2to1ParamWord {
    val name = "*"
    def eval() {
      prepare()
      stack = (left(mc) * right) :: stack.tail
    }
  }

  object DividedBy extends Abstract2to1ParamWord {
    val name = "/"
    def eval() {
      prepare()
      stack = (left(mc) / right) :: stack.tail
    }
  }

  object Quit extends AbstractWord {
    val name = "quit"
    def eval() {
      Console.out.print("Live long and prosper! ")
      System.exit(0)
    }
  }

  object Mode extends Enumeration {
    val interpret, compile = Value
  }

  object Colon extends AbstractWord {
    val name = ":"
    def eval() {
      mode = Mode.compile
    }
    override def compile() {

    }
  }

  object SemiColon extends AbstractWord {
    val name = ";"
    def eval() {
    }
    override def compile() {
      mode = Mode.interpret
      val newName = compilationStack.head
      val words = compilationStack.tail
      dict += (newName -> new Word(name = newName, words = words))
      compilationStack = List[String]()
      nameDefinition = true
    }
  }
}

Scala and FORTH - part 2

So far we have only implemented primitive words, and in a typical Forth implementation these words are assembly language constructs. We've use Scala instead, of course. But we need to support creating programs, not just using what's there.

In Forth you program by compiling new words. A word definition starts with a ":" and ends with a ";". E.g. ": ANewWord quit ;" would define a new word called ANewWord, which would execute the existing word "quit" when invoked.

So what do we need to implement compilation? We'll start collecting definitions when ":" is encountered, and compile the gathered data when ";" ends the definition. So we need a place to put stuff until the actual compilation:

  var compilationStack = List[String]()

We also need to know whether to execute or compile a word, ":" will enter compilation mode, and ";" will switch back to interpret mode:

  var mode = Mode.interpret

  object Mode extends Enumeration {
    val interpret, compile = Value
  }

Finally, our implementation will need to be able to differentiate between name and implementation:

  var nameDefinition = true

With that out of the way, we need to add compile time behaviour to the words:

  abstract class AbstractWord {
    val name:String
    def eval
    def compile {
      compilationStack = compilationStack ::: name :: Nil
    }
  }

The default compile implementation is to just add its own name to the compilation stack. ":" just needs to switch mode, while ";" also needs to compile the compilation stack. It does this by using the first stack element as name, and the rest as a list of word names to be executed:

  object Colon extends AbstractWord {
    val name = ":"
    def eval {
      mode = Mode.compile
    }
    override def compile {
    }
  }

   object SemiColon extends AbstractWord {
     val name = ";"
     def eval {
     }
     override def compile {
       mode = Mode.interpret
       val newName = compilationStack.head
       val words = compilationStack.tail
       dict += (newName -> new Word(name = newName, words = words))
       compilationStack = List[String]()
       nameDefinition = true
     }
   }

In the main loop we need to handle compilation:

  def main(args: Array[String]) {
    ...
      for (item <- items) {
        if (mode == Mode.interpret) {
          ...
          }
        } else {
          if(nameDefinition) {
            compilationStack = item :: compilationStack
            nameDefinition = false
          } else {
            dict.get(item) match {
              case Some(s:AbstractWord) => s.compile
              case None => Console.print("Skipping " + item)
            }
          }
        }
      }
      ...
  }

  def initDict {
    ...
    dict += (":" -> Colon)
    dict += (";" -> SemiColon)
  }

And that's it! A minimal Forth compiler ready to go:

Welcome to scalaFORTH
> : .. . . ;
 ok
> 1 2 .. 1 2 . .
2 1 2 1  ok
>

Next time we'll tackle numbers at compile time. As of now scalaFORTH looks like this:

object FORTHv3 {
  var stack = List[BigDecimal]()
  var dict = Map[String, AbstractWord]()
  var compilationStack = List[String]()
  var mode = Mode.interpret
  var nameDefinition = true

  def main(args: Array[String]) {
    initDict
    Console.println("Welcome to scalaFORTH")
    Console.print("> ")
    while (true) {
      val line = Console.in.readLine
      val items = line.split(' ')
      for (item <- items) {
        if (mode == Mode.interpret) {
          dict.get(item) match {
            case Some(s:AbstractWord) => s.eval
            case None => stack = BigDecimal.apply(item) :: stack
          }
        } else {
          if(nameDefinition) {
            compilationStack = item :: compilationStack
            nameDefinition = false
          } else {
            dict.get(item) match {
              case Some(s:AbstractWord) => s.compile
              case None => Console.print("Skipping " + item)
            }
          }
        }
      }
      Console.println(" ok")
      Console.print("> ")
    }
  }

  def initDict {
    dict += ("." -> Dot)
    dict += ("quit" -> Quit)
    dict += (":" -> Colon)
    dict += (";" -> SemiColon)
  }

  abstract class AbstractWord {
    val name:String
    def eval
    def compile {
      compilationStack = compilationStack ::: name :: Nil
    }
  }

  class Word(var immediate: Boolean = false, val name: String, val words:List[String] = List()) extends AbstractWord {
    def eval {
      for (word <- words) {
        dict.get(word) match {
          case Some(s:AbstractWord) => s.eval
          case None => Console.err.println("Word not found: " + word)
        }
      }
    }
  }

  object Dot extends AbstractWord {
    val name = "."
    def eval {
      Console.out.print(stack.head + " ")
      stack=stack.tail
    }
  }

  object Quit extends AbstractWord {
    val name = "quit"
    def eval {
      Console.out.print("Live long and prosper! ")
      System.exit(0)
    }
  }

  object Mode extends Enumeration {
    val interpret, compile = Value
  }

  object Colon extends AbstractWord {
    val name = ":"
    def eval {
      mode = Mode.compile
    }
    override def compile {

    }
  }

  object SemiColon extends AbstractWord {
    val name = ";"
    def eval {
    }
    override def compile {
      mode = Mode.interpret
      val newName = compilationStack.head
      val words = compilationStack.tail
      dict += (newName -> new Word(name = newName, words = words))
      compilationStack = List[String]()
      nameDefinition = true
    }
  }
}

Scala and FORTH - part 1

I was sorting my books the other day, and I came across an old copy of a FORTH manual. Yes we're talking FORTH from the dark ages, before all caps fell out of fashion (the manual was printed in 1981 to be more precise). It must have been more than 20 years since I programmed in FORTH.

Why not see how far I can get writing my own Forth using Scala, I thought to myself. So let's start! But before we start beware that this is a purely iterative process. I have no idea if I'll paint myself into a corner eventually. (It does seems likely though...) Enter scalaFORTH, which will be loosely based on figFORTH.

Forth is interpreted, and is used interactively, so we'll start with a prompt within a loop:

object FORTHv1 {

  def main(args: Array[String]) {
    Console.println("Welcome to scalaFORTH")
    Console.print("> ")
    while (true) {
      val line = Console.in.readLine
      if (line == "quit") System.exit(0)
      Console.println(" ok")
      Console.print("> ")
    }
  }
}

As you can see, quit is the command to exit the interpreter. Traditionally Forth is case insensitive, but the aim of scalaForth is not to be a faithful, to-the-letter, implemetation of figFORTH, just an attempt to get that warm feeling you get when programming Forth.

One fairly unique aspect of Forth is that it's stack based, and functions are typically performed on arguments on the stack, like on HP calculators. So "function(1, 2)" would look like "1 2 function" in Forth. One of the simplest and most used words is "." (dot). "." pops the top element from the stack and prints it on the console. A word is the Forth equivalent of a subroutine, method or what have you. Words are stored in a dictionary in a compiled format so that they can execute directly.

object FORTHv2 {
  var stack = List[BigDecimal]()
  var dict = Map[String, AbstractWord]()

  def main(args: Array[String]) {
    initDict
    Console.println("Welcome to scalaFORTH")
    Console.print("> ")
    while (true) {
      val line = Console.in.readLine
//      if (line == "quit") System.exit(0)
      val items = line.split(' ')
      for (item <- items) {
        dict.get(item) match {
          case Some(s:AbstractWord) => s.eval
          case None => stack = BigDecimal.apply(item) :: stack
        }
      }
      Console.println(" ok")
      Console.print("> ")
    }
  }

  def initDict {
    dict += ("." -> Dot)
    dict += ("quit" -> Quit)
  }

  abstract class AbstractWord {
    def eval
  }

  object Dot extends AbstractWord {
    def eval {
      Console.out.print(stack.head + " ")
      stack=stack.tail
    }
  }

  object Quit extends AbstractWord {
    def eval {
      Console.out.print("Live long and prosper! ")
      System.exit(0)
    }
  }
}

A stack and a dictionary has been added, and also the base class for all words. If the interpreter fails to look up a word, it will try to parse it as a number instead. Note that decimal numbers are used, figForth only supported integers.

Now that we have a dictionary, we can put "quit" there, and we also added ".". Run scalaFORTH and try some numbers!

Welcome to scalaFORTH
> 1 2 3
 ok
> . . .
3 2 1  ok
> quit
Live long and prosper!

In the next part I'll try to get compilation to work.