On Smalltalk

Finally, an explanation of how the MessageTally profiler in Squeak works. I’ve been meaning to ask someone for an explanation of what all this info means, but haven’t quite got around to it yet. Thank you Andreas! Quoting here in case that site becomes unavailable…

MessageTally
------------

The primary tool to measure performance, both for Squeak in general as
well as for Croquet, is MessageTally. MessageTally acts on a particular
expression (MessageTally spyOn:["your expression here"]) and provides
the following information:
a) A hierarchy showing how much time was spent where in the computation.
b) A list showing which amount of time was spent in which leaf nodes
c) Memory statistics, incl. the growth rate, garbage collection info etc.

MessageTally uses a technique known as “pc-sampling”. What that means is
that a high-priority process is started which (based on a timer) samples
the call stack of the process and allocates a time value (typically
whatever it’s using for sampling).

It is important to notice that this is a statistical measure - given a
large enough number of samples, the reported result will be
statistically valid. On the other hand, small numbers of samples are
typically statistically invalid - a single garbage collection can lead
to a major change in an otherwise insignificant part of the computation.

Here is an example:

    MessageTally spyOn:[100 raisedTo: 1000].

resulted in the following output:

**Tree**
100.0% {3ms} SmallInteger(Number)>>raisedTo:
   100.0% {3ms} SmallInteger(Number)>>raisedToInteger:
     50.0% {2ms} LargePositiveInteger>>*
     50.0% {2ms} primitives

These results claim that we’re spending 50% of the overall time in
Multiplying large integers. Which is completely bogus since we have only
two samples (the default sampling rate is 1ms). If we run this for a
longer period of time, like here:

   MessageTally spyOn:[1000 timesRepeat:[100 raisedTo: 1000]]

**Tree**
100.0% {1007ms} SmallInteger(Number)>>raisedTo:
   99.9% {1006ms} SmallInteger(Number)>>raisedToInteger:
     96.8% {975ms} primitives
     3.0% {30ms} LargePositiveInteger>>*

We see that indeed, we only spend roughly 3% in multiplying large
integers. The other 97% are spent in “primitives” which, unfortunately,
are not broken out separately in the measures (however, if such a
measure is critical, then the primitives can to be factored into
separate methods which then call the primitives themselves - this allows
message tally to “see” the method frames and report the usage accordingly).

In a more complex situation, the percentage tree is typically useful to
figure out roughly the areas in which time is spent which can then be
measured individually.

**Leaves**
———-

The **Leaves* reported by MessageTally is the amount of time spent in a
method WITHOUT the time spent in the methods called from that method. In
our above example the leaves are reported as:

**Leaves**
96.8% {975ms} SmallInteger(Number)>>raisedToInteger:

which is the overall time spent in Number>>raisedToInteger: (1006ms)
minus the time spent in LargePositiveInteger>>* (30ms). If a method
shows up in the leaves it typically means that this method is
computationally expensive or just gets called a large number of times.

**Memory**
———-

The **Memory** statistics shown in MessageTally provide information
about how various memory regions have changed:

- old: Describes the “old space” in memory. This is the portion that
will not be included in incremental garbage collection but only during a
full garbage collection. See also the “tenure” information below.
Extensively growing old space typically means that there is a problem
with the allocation patterns or garbage collector settings.

- young: Describes the “young space” in memory, e.g., the region handled
by the incremental garbage collector. Changes in young space are usually
not relevant.

- used: Total amount of used memory.

- free: Total amount of unused memory.

**GCs**
——-

The *GCs* statistics provide information about the garbage collector:

- full: The number of “full” garbage collections and time spent in
those. Automatic full garbage collections should be VERY rare, they are
a sign that you’re allocating huge amounts of memory repeatedly. Note
that at times these garbage collections are manually triggered though
(like in the checkpointing process) and a normal effect of the operation.

- incr: The number of “incremental” garbage collections and time spent
in those. Generally speaking, IGCs should be quick (avg. < 2ms) and
frequent (several times a second). However, the total time spent in IGCs
should generally be less than 10%, otherwise this is a sign of a problem
with the allocation patterns. If the time spent in IGCs exceeds 25%
something is *definitely* wrong.

- tenures: Tenuring occurs when the number of surviving objects in young
space exceeds a certain threshold. In this case, the young space
boundary is increased (which adds to the size of "old space" mentioned
above). Tenuring typically means that the working set of the application
hasn't been reached. For example, in a space construction we would
expect frequent tenuring until the space is fully constructed. Once the
working set has been reached, tenuring should be rare to non-existent.
Frequent tenuring in such cases means that the garbage collection
parameters need to be adjusted.

- root table overflows: Root table overflows describe the (rare) case
that the number of "roots" for the incremental garbage collector will
overflow the internal table. This will force an immediate garbage
collection plus tenuring. The measure is provided in order to be able to
find such rare cases (which otherwise leave you wondering why the system
is running full GCs all the time for no apparent reason)

Multiple processes
------------------
Historically, MessageTally measured and reported only the call stack of
the current process. This had the disadvantage that if time was spent in
a different process, it would be attributed to a bogus frame in current
thread. For Croquet, I have changes this such that *all* processes are
reported in order to be able to see "what else" is going on.

For example, if we measure an expression like here:

   MessageTally spyOn:[(Delay forSeconds: 5) wait]

we will find that all of the time is reported here:

**Tree**
99.5% {4975ms} ProcessorScheduler class>>startUp
   99.5% {4975ms} ProcessorScheduler class>>idleProcess

The idle process is the process that is being activated when no other
activity occurs (the implementation of the idle process requests the VM
to sleep for a millisecond so that the VM isn’t running a busy).
Generally, time reported in idleProcess is time spent “doing nothing”
(e.g., waiting for some activity).

The other relevant system process that may show up is the finalization
process. If the finalization process shows up, it means we’re having a
problem with too many weak references being finalized. This has been a
*big* problem in the past, so keep an eye on it.

Cheers,
   - Andreas

Tags: General, Performance, Profiling, Smalltalk, Squeak

Seaside, unlike most frameworks, has no concept of page templates. This may at first seem daunting, but this is one of its greatest strengths; you will soon learn to love it.

HTML templates suffer from many problems, chief among them a total lack of abstraction and ability to reuse common pieces of HTML within a page without simply copying it. Many frameworks offer some form of user control in an attempt to remedy this, but these invariable fail or become hard to use cleanly because of limitation in the host frameworks ability to wire, coordinate, and maintain state of this component tree, while keeping the HTML in sync with the latest changes made by a designer.

Templates mislead designers into thinking HTML is for presentation, rather than structure to be marked up and lain out using CSS. Putting HTML squarely in the hands of programmers, forces a better separation between developer and designer, between functionality and beauty.

The WARenderCanvas is the class within the Seaside framework which contains a domain specific dialect of Smalltalk allowing the programmer to express HTML directly in native Smalltalk code, seamlessly blending the generation of the HTML with ordinary Smalltalk code, giving the developer the full abstraction capabilities of Smalltalk.

The developer can now freely break up large chunks of HTML into smaller, more general, reusable, and manageable chunks, and take advantage of Smalltalk’s amazing development environment to do so.

To create a page in Seaside, one simply subclasses WAComponent and writes a renderContentOn: method like so…

renderContentOn: html
    html text: 'Hello World'

html, is an instance of the Canvas Renderer. In the spirit of learning by doing, let’s play around a bit and enhance this example.

renderContentOn: html
    html div: 'Hello World'

renders <div>Hello world</div> and

renderContentOn: html
    html div class: #title; with: 'Hello World'

renders <div class=”title”>Hello world</div> and

renderContentOn: html
    html div
         class: #title;
         with: [html span: 'Hello World']

renders <div class=”title”><span>Hello world</span></div>

From these few examples, the basic pattern of the rendering canvas should be clear. The canvas contains a factory method for creating HTML tags, tags can be sent messages to set their attributes, which usually translate directly to their counterpart HTML attributes.

Smalltalkers dislike abbreviations so where appropriate, some attributes or tag names are expanded into full words, such as “anchor” rather than simply “a”. I have yet to see any such changes I didn’t fully agree with.

There is one special method in the canvas API, this is the with: method. This method must be the last method sent to a tag, for it contains the contents of the tag, calling it will stop the tag from accepting any further attributes. Using Smalltalk’s blocks “[]“, cascade operator “;”, and tag objects, you can fully express any HTML tag.

I lied, there are two special methods in the canvas API, text: is a synonym for with: on the anchor tag.

renderContentOn: html
    html div
         class: #title;
         with: [html anchor
                     callback:[Smalltalk beep];
                     text: ‘Hello World’]

callback: is the message you will enjoy the most when working in Seaside. This message is central to the magic of Seaside, and the key thing that makes programming in Seaside vastly different from programming in any other framework.

By sending a block of code to callback: on the anchor tag, I’ve attached that code as the action to be taken when the anchor is clicked, without thinking about state, or embedding any special tokens into a special URL.

While some frameworks have event systems that allow similar code, few do it with closures that capture state from their local environment, and don’t lose state between postbacks of the web page between user clicks. This allows Seaside programming to be very similar to traditional desktop programming, without all the hassles of state management the web introduced.

No more embedding parameters into URL’s, no more cookies, no more loading necessary state from hidden form elements, or from the database at each post back, just to continue where you left off after the last user action. The URL, cookies, application and session state, are all still available, should you need to use them directly, or accept data posted from other non Seaside applications, but you don’t “have” to use them. The bulk of programming in Seaside is done via callbacks, and passing of objects between components via call: and answer:, freeing one to focus on the problem rather than hacking around the framework.

Remember when I said HTML and real code could be mixed seamlessly, try this in your standard web framework.

renderContentOn: html
    10 timesRepeat:[
        html
            heading: Random sentence;
            paragraph: Random paragraph]

which generates 10 paragraphs of random Latin text, each with a random Latin heading. If you build a lot of prototypes, you might find something like this useful, so useful in fact, that you decide to extend seaside to include it as part of the rendering canvas, a common technique in Smalltalk and Seaside. So you generalize this method into

WARenderCanvas>>randomContentSections: aNum
    aNum timesRepeat:[
        self
            heading: Random sentence;
            paragraph: Random paragraph]

either by subclassing the render canvas, or simply adding an extension method to the existing one. Next time you can simply write

renderContentOn: html
    html randomConentSections: 10.

and not have to think about how it was implement, only that it works. In short, Seaside rocks, it’s the best web framework out today. There’s much talk about Ruby on Rails, but Rails is just today’s framework done right, Seaside is tomorrow’s framework today!

Tags: General, Programming, Ruby, Seaside, Smalltalk