Category: Programming

Lambda Expressions Backported to Java 7, 6 and 5

Do you want to use lambda expressions already today, but you are forced to use Java and a stable JRE in production? Now that’s possible with Retrolambda, which will take bytecode compiled with Java 8 and convert it to run on Java 7, 6 and 5 runtimes, letting you use lambda expressions andmethod references on those platforms. It won’t give you the improved Java 8 Collections API, but fortunately there are multiple alternative libraries which will benefit from lambda expressions.

Behind the Scenes

A couple of days ago in a café it popped into my head to find out whether somebody had made this already, but after speaking into the air, I did it myself over a weekend.

The original plan of copying the classes from OpenJDK didn’t work (LambdaMetafactory depends on some package-private classes and would have required modifications), but I figured out a better way to do it without additional runtime dependencies.

Retrolambda uses a Java agent to find out what bytecode LambdaMetafactory generates dynamically, and saves it as class files, after which it replaces the invokedynamic instructions to instantiate those classes directly. It also changes some private synthetic methods to be package-private, so that normal bytecode can access them without method handles.

After the conversion you’ll have just a bunch of normal .class files – but with less typing.

P.S. If you hear about experiences of using Retrolambda for Android development, please leave a comment.


5 Coding Hacks to Reduce GC Overhead

In this post we’ll look at five ways in roomates efficient coding we can use to help our garbage collector CPU spend less time allocating and freeing memory, and reduce GC overhead. Often Long GCs can lead to our code being stopped while memory is reclaimed (AKA “stop the world”). Duke_GCPost

Some background

The GC is built to handle large amounts of allocations of short-lived objects (think of something like rendering a web page, where most of the objects allocated Become obsolete once the page is served).

The GC does this using what’s called a “young generation” – a heap segment where new objects are allocated. Each object has an “age” (placed in the object’s header bits) defines how many roomates collections it has “survived” without being reclaimed. Once a certain age is reached, the object is copied into another section in the heap called a “survivor” or “old” generation.

The process, while efficient, still comes at a cost. Being Able to reduce the number of temporary allocations can really help us increase of throughput, especially in high-scale applications.

Below are five ways everyday we can write code that is more memory efficient, without having to spend a lot of time on it, or reducing code readability.

1. Avoid implicit Strings

Strings are an integral part of almost every structure of data we manage. Being much heavier than other primitive values, they have a much stronger impact on memory usage.

One of the most important things to note is that Strings are immutable. They can not be modified after allocation. Operators such as “+” for concatenation actually allocate a new String containing the contents of the strings being joined. What’s worse, is there’s an implicit StringBuilder object that is allocated to actually do the work of combining them.

For example –

1
a = a + b; / / a and b are Strings
The compiler generates code comparable behind the scenes:

1
StringBuilder temp = new StringBuilder (a).
2
temp.append (b);
3
a = temp.toString () / / a new string is allocated here.
4
/ / The previous “a” is now garbage.
But it gets worse.

Let’s look at this example –

1
String result = foo () + arg;
2
result + = boo ();
3
System.out.println (“result =” + result);
In this example we have 3 StringBuilders allocated in the background – one for each plus operation, and two additional Strings – one to hold the result of the second assignment and another to hold the string passed into the print method. That’s 5 additional objects in what would otherwise Appear to be a pretty trivial statement.

Think about what happens in real-world scenarios such as generating code a web page, working with XML or reading text from a file. Within a nested loop structures, you could be looking at Hundreds or Thousands of objects that are implicitly allocated. While the VM has Mechanisms to deal with this, it comes at a cost – one paid by your users.

The solution: One way of reducing this is being proactive with StringBuilder allocations. The example below Achieves the same result as the code above while allocating only one StringBuilder and one string to hold the final result, instead of the original five objects.

1
StringBuilder value = new StringBuilder (“result =”);
2
value.append (foo ()). append (arg). append (boo ());
3
System.out.println (value);
By being mindful of the way Strings are implicitly allocated and StringBuilders you can materially reduce the amount of short-term allocations in high-scale code locations.

2. List Plan capacities

Dynamic collections such as ArrayLists are among the most basic dynamic structures to hold the data length. ArrayLists and other collections such as HashMaps and implemented a Treemaps are using the underlying Object [] arrays. Like Strings (Themselves wrappers over char [] arrays), arrays are also immutable. Becomes The obvious question then – how can we add / put items in their collections if the underlying array’s size is immutable? The answer is obvious as well – by allocating more arrays.

Let’s look at this example –

1
List <Item> <Item> items = new ArrayList ();
2

3
for (int i = 0; i <len; i + +)
4
{
5
Item item = readNextItem ();
6
items.add (item);
7
}
The value of len Determines the ultimate length of items once the loop finishes. This value, however, is unknown to the constructor of the ArrayList roomates allocates a new Object array with a default size. Whenever the internal capacity of the array is exceeded, it’s replaced with a new array of sufficient length, making the previous array of garbage.

If you’re executing the loop Welcome to Thunderbird times you may be forcing a new array to be allocated and a previous one to be collected multiple times. For code running in a high-scale environment, these allocations and deallocations are all deducted from your machine’s CPU cycles.
%0


Integrating C++ with QML

Introduction

Qt Quick’s QML language makes it easy to do many things, especially fancy animated user interfaces. However, some things either can’t be done or are not suitable for implementing in QML, such as:

  1. Getting access to functionality outside of the QML/JavaScript environment.
  2. Implementing performance critical functions where native code is desired for efficiency.
  3. Large and/or complex non-declarative code that would be tedious to implement in JavaScript.

As we’ll see, Qt makes it quite easy to expose C++ code to QML. In this blog post I will show an example of doing this with a small but functional application.

The example is written for Qt 5 and uses the Qt Quick Components so you will need at least Qt version 5.1.0 to run it.

Overview

To expose a C++ type having properties, methods, signals, and/or slots to the QML environment, the basic steps are:

  1. Define a new class derived from QObject.
  2. Put the Q_OBJECT macro in the class declaration to support signals and slots and other services of the Qt meta-object system.
  3. Declare any properties using the Q_PROPERTY macro.
  4. Call qmlRegisterType() in your C++ main program to register the type with the Qt Quick engine.

For all the details I refer you to the Qt documentation section Exposing Attributes of C++ Types to QML and the Writing QML Extensions with C++ tutorial.

Ssh Key Generator

For our code example, we want a small application that will generate ssh public/private key pairs using a GUI. It will present the user with controls for the appropriate options and then run the program ssh-keygen to generate the key pair.

I implemented the user interface using the new Qt Quick Controls since it was intended as a desktop application with a desktop look and feel. I initially developed the UX entirely by running the qmlscene program directly on the QML source.

The UI prompts the user for the key type, the file name of the private key to generate and an optional pass phrase, which needs to be confirmed.

The C++ Class

Now that have the UI, we will want to implement the back end functionality. You can’t invoke an external program directly from QML so we have to write it in C++ (which is the whole point of this example application).

First, we define a class that encapsulates the key generation functionality. It will be exposed as a new class KeyGenerator in QML. This is done in the header file KeyGenerator.h below.

#ifndef KEYGENERATOR_H
#define KEYGENERATOR_H

#include <QObject>
#include <QString>
#include <QStringList>

// Simple QML object to generate SSH key pairs by calling ssh-keygen.

class KeyGenerator : public QObject
{
    Q_OBJECT
    Q_PROPERTY(QString type READ type WRITE setType NOTIFY typeChanged)
    Q_PROPERTY(QStringList types READ types NOTIFY typesChanged)
    Q_PROPERTY(QString filename READ filename WRITE setFilename NOTIFY filenameChanged)
    Q_PROPERTY(QString passphrase READ filename WRITE setPassphrase NOTIFY passphraseChanged)

public:
    KeyGenerator();
    ~KeyGenerator();

    QString type();
    void setType(const QString &t);

    QStringList types();

    QString filename();
    void setFilename(const QString &f);

    QString passphrase();
    void setPassphrase(const QString &p);

public slots:
    void generateKey();

signals:
    void typeChanged();
    void typesChanged();
    void filenameChanged();
    void passphraseChanged();
    void keyGenerated(bool success);

private:
    QString _type;
    QString _filename;
    QString _passphrase;
    QStringList _types;
};
#endif

Next, we need to derive our class from QObject. We declare any properties that we want and the associated methods. Notify methods become signals. In our case, we want to have properties for the selected key type, the list of all valid ssh key types, file name and pass phrase. I arbitrarily made the key type a string. It could have been an enumerated type but it would have made the example more complicated.

Incidentally, a new feature of the Q_PROPERTY macro in Qt 5.1.0 is the MEMBER argument. It allows specifying a class member variable that will be bound to a property without the need to implement the setter or getter functions. That feature was not used here.

We declare methods for the setters and getters and for signals. We also declare one slot called generateKey(). These will all be available to QML. If we wanted to export a regular method to QML, we could mark it with Q_INVOCABLE. In this case I decided to make generateKey() a slot since it might be useful in the future but it could have just as easily been an invocable method.

Finally, we declare any private member variables we will need.

C++ Implementation

Now let’s look at the implementation in KeyGenerator.cpp. Here is the source code:

#include <QFile>
#include <QProcess>
#include "KeyGenerator.h"

KeyGenerator::KeyGenerator()
    : _type("rsa"), _types{"dsa", "ecdsa", "rsa", "rsa1"}
{
}

KeyGenerator::~KeyGenerator()
{
}

QString KeyGenerator::type()
{
    return _type;
}

void KeyGenerator::setType(const QString &t)
{
    // Check for valid type.
    if (!_types.contains(t))
        return;

    if (t != _type) {
        _type = t;
        emit typeChanged();
    }
}

QStringList KeyGenerator::types()
{
    return _types;
}

QString KeyGenerator::filename()
{
    return _filename;
}

void KeyGenerator::setFilename(const QString &f)
{
    if (f != _filename) {
        _filename = f;
        emit filenameChanged();
    }
}

QString KeyGenerator::passphrase()
{
    return _passphrase;
}

void KeyGenerator::setPassphrase(const QString &p)
{
    if (p != _passphrase) {
        _passphrase = p;
        emit passphraseChanged();
    }
}

void KeyGenerator::generateKey()
{
    // Sanity check on arguments
    if (_type.isEmpty() or _filename.isEmpty() or
        (_passphrase.length() > 0 and _passphrase.length() < 5)) {
        emit keyGenerated(false);
        return;
    }

    // Remove key file if it already exists
    if (QFile::exists(_filename)) {
        QFile::remove(_filename);
    }

    // Execute ssh-keygen -t type -N passphrase -f keyfileq
    QProcess *proc = new QProcess;
    QString prog = "ssh-keygen";
    QStringList args{"-t", _type, "-N", _passphrase, "-f", _filename};
    proc->start(prog, args);
    proc->waitForFinished();
    emit keyGenerated(proc->exitCode() == 0);
    delete proc;
}

The constructor initializes some of the member variables. For fun, I used the new initializer list feature of C++11 to initialize the _types member variable which is of type QStringList. The destructor does nothing, at least for now, but is there for completeness and future expansion.

Getter functions like type() simply return the appropriate private member variable. Setters set the appropriate variables, taking care to check that the new value is different from the old one and if so, emitting the appropriate signal. As always, please note that signals are created by the Meta Object Compiler and do not need to be implemented, only emitted at the appropriate times.

The only non-trivial method is the slot generateKey(). It does some checking of arguments and then creates a QProcess to run the external ssh-keygen program. For simplicity and because it typically executes quickly, I do this synchronously and block on it to complete. When done, we emit a signal that has a boolean argument that indicates the key was generated and whether it succeeded or not.

QML Code

Now let’s look at the QML code in main.qml:

// SSH key generator UI

import QtQuick 2.1
import QtQuick.Controls 1.0
import QtQuick.Layouts 1.0
import QtQuick.Dialogs 1.0
import com.ics.demo 1.0

ApplicationWindow {
    title: qsTr("SSH Key Generator")

    statusBar: StatusBar {
    RowLayout {
        Label {
            id: status
            }
        }
    }

    width: 369
    height: 166

    ColumnLayout {
        x: 10
        y: 10

        // Key type
        RowLayout {
            Label {
                text: qsTr("Key type:")
            }
            ComboBox {
                id: combobox
                Layout.fillWidth: true
                model: keygen.types
                currentIndex: 2
            }
        }

        // Filename
        RowLayout {
            Label {
                text: qsTr("Filename:")
            }
            TextField {
                id: filename
                implicitWidth: 200
                onTextChanged: updateStatusBar()
            }
            Button {
                text: qsTr("&Browse...")
                onClicked: filedialog.visible = true
            }
        }

        // Passphrase
        RowLayout {
            Label {
                text: qsTr("Pass phrase:")
            }
            TextField {
                id: passphrase
                Layout.fillWidth: true
                echoMode: TextInput.Password
                onTextChanged: updateStatusBar()
            }

        }

        // Confirm Passphrase
        RowLayout {
            Label {
                text: qsTr("Confirm pass phrase:")
            }
            TextField {
                id: confirm
                Layout.fillWidth: true
                echoMode: TextInput.Password
                onTextChanged: updateStatusBar()
            }
        }

        // Buttons: Generate, Quit
        RowLayout {
            Button {
                id: generate
                text: qsTr("&Generate")
                onClicked: keygen.generateKey()
            }
            Button {
                text: qsTr("&Quit")
                onClicked: Qt.quit()
            }
        }

    }

    FileDialog {
        id: filedialog
        title: qsTr("Select a file")
        selectMultiple: false
        selectFolder: false
        nameFilters: 
        selectedNameFilter: "All files (*)"
        onAccepted: {
            filename.text = fileUrl.toString().replace("file://", "")
        }
    }

    KeyGenerator {
        id: keygen
        filename: filename.text
        passphrase: passphrase.text
        type: combobox.currentText
        onKeyGenerated: {
            if (success) {
                status.text = qsTr('<font color="green">Key generation succeeded.</font>')
            } else {
                status.text = qsTr('<font color="red">Key generation failed</font>')
            }
        }
    }

    function updateStatusBar() {
        if (passphrase.text != confirm.text) {
            status.text = qsTr('<font color="red">Pass phrase does not match.</font>')
            generate.enabled = false
        } else if (passphrase.text.length > 0 && passphrase.text.length < 5) {
            status.text = qsTr('<font color="red">Pass phrase too short.</font>')
            generate.enabled = false
        } else if (filename.text == "") {
            status.text = qsTr('<font color="red">Enter a filename.</font>')
            generate.enabled = false
        } else {
            status.text = ""
            generate.enabled = true
        }
    }

    Component.onCompleted: updateStatusBar()
}

The preceding code is a little long, however, much of the work is laying out the GUI components. The code should be straightforward to follow.

Note that we import com.ics.demo version 1.0. We’ll see where this module name comes from shortly. This makes a new QML type KeyGeneratoravailable and so we declare one. We have access to it’s C++ properties as QML properties, can call it’s methods and act on signals like we do withonKeyGenerated.

A more complete program should probably do a little more error checking and report meaningful error messages if key generation fails (we could easily add a new method or property for this). The UI layout could also be improved to make it properly resizable.

Our main program is essentially a wrapper like qmlscene. All we need to do to register our type with the QML engine is to call:

    qmlRegisterType<KeyGenerator>("com.ics.demo", 1, 0, "KeyGenerator");

This makes the C++ type KeyGenerator available as the QML type KeyGenerator in the module com.ics.demo version 1.0 when it is imported.

Typically, to run QML code from an executable, in the main program you would create a QGuiApplication and a QQuickView. Currently, to use the Qt Quick Components there is some additional work needed if the top level element is an ApplicationWindow or Window. You can look at the source code to see how I implemented this. I basically stripped down the code from qmlscene to the minimum of what was needed for this example.

Here is the full listing for the main program, main.cpp:

#include <QApplication>
#include <QObject>
#include <QQmlComponent>
#include <QQmlEngine>
#include <QQuickWindow>
#include <QSurfaceFormat>
#include "KeyGenerator.h"

// Main wrapper program.
// Special handling is needed when using Qt Quick Controls for the top window.
// The code here is based on what qmlscene does.

int main(int argc, char ** argv)
{
    QApplication app(argc, argv);

    // Register our component type with QML.
    qmlRegisterType<KeyGenerator>("com.ics.demo", 1, 0, "KeyGenerator");

    int rc = 0;

    QQmlEngine engine;
    QQmlComponent *component = new QQmlComponent(&engine);

    QObject::connect(&engine, SIGNAL(quit()), QCoreApplication::instance(), SLOT(quit()));

    component->loadUrl(QUrl("main.qml"));

    if (!component->isReady() ) {
        qWarning("%s", qPrintable(component->errorString()));
        return -1;
    }

    QObject *topLevel = component->create();
    QQuickWindow *window = qobject_cast<QQuickWindow *>(topLevel);

    QSurfaceFormat surfaceFormat = window->requestedFormat();
    window->setFormat(surfaceFormat);
    window->show();

    rc = app.exec();

    delete component;
    return rc;
}

In case it is not obvious, when using a module written in C++ with QML you cannot use the qmlscene program to execute your QML code because the C++ code for the module will not be linked in. If you try to do this you will get an error message that the module is not installed.


OpenMP 4.0 Specifications Released

The OpenMP 4.0 API Specification is released with Significant New Standard Features

The OpenMP 4.0 API supports the programming of accelerators, SIMD programming, and better optimization using thread affinity

The OpenMP Consortium has released OpenMP API 4.0, a major upgrade of the OpenMP API standard language specifications. Besides several major enhancements, this release provides a new mechanism to describe regions of code where data and/or computation should be moved to another computing device.

Bronis R. de Supinski, Chair of the OpenMP Language Committee, stated that “OpenMP 4.0 API is a major advance that adds two new forms of parallelism in the form of device constructs and SIMD constructs. It also includes several significant extensions for the loop-based and task-based forms of parallelism already supported in the OpenMP 3.1 API.

The 4.0 specification is now available on the 

Standard for parallel programming extends its reach

With this release, the OpenMP API specifications, the de-facto standard for parallel programming on shared memory systems, continues to extend its reach beyond pure HPC to include DSPs, real time systems, and accelerators. The OpenMP API aims to provide high-level parallel language support for a wide range of applications, from automotive and aeronautics to biotech, automation, robotics and financial analysis.

New features in the OpenMP 4.0 API include:

· Support for accelerators. The OpenMP 4.0 API specification effort included significant participation by all the major vendors in order to support a wide variety of compute devices. OpenMP API provides mechanisms to describe regions of code where data and/or computation should be moved to another computing device. Several prototypes for the accelerator proposal have already been implemented.

· SIMD constructs to vectorize both serial as well as parallelized loops. With the advent of SIMD units in all major processor chips, portable support for accessing them is essential. OpenMP 4.0 API provides mechanisms to describe when multiple iterations of the loop can be executed concurrently using SIMD instructions and to describe how to create versions of functions that can be invoked across SIMD lanes.

· Error handling. OpenMP 4.0 API defines error handling capabilities to improve the resiliency and stability of OpenMP applications in the presence of system-level, runtime-level, and user-defined errors. Features to abort parallel OpenMP execution cleanly have been defined, based on conditional cancellation and user-defined cancellation points.

· Thread affinity. OpenMP 4.0 API provides mechanisms to define where to execute OpenMP threads. Platform-specific data and algorithm-specific properties are separated, offering a deterministic behavior and simplicity in use. The advantages for the user are better locality, less false sharing and more memory bandwidth.

· Tasking extensions. OpenMP 4.0 API provides several extensions to its task-based parallelism support. Tasks can be grouped to support deep task synchronization and task groups can be aborted to reflect completion of cooperative tasking activities such as search. Task-to-task synchronization is now supported through the specification of task dependency.

· Support for Fortran 2003. The Fortran 2003 standard adds many modern computer language features. Having these features in the specification allows users to parallelize Fortran 2003 compliant programs. This includes interoperability of Fortran and C, which is one of the most popular features in Fortran 2003.

· User-defined reductions. Previously, OpenMP API only supported reductions with base language operators and intrinsic procedures. With OpenMP 4.0 API, user-defined reductions are now also supported.

· Sequentially consistent atomics. A clause has been added to allow a programmer to enforce sequential consistency when a specific storage location is accessed atomically.

This represents collaborative work by many of the brightest in industry, research, and academia, building on the consensus of 26 members. We strive to deliver high-level parallelism that is portable across 3 widely-implemented common General Purpose languages, productive for HPC and consumers, and delivers highly competitive performance. I want to congratulate all the members for coming together to create such a momentous advancement in parallel programming, under such tight constraints and industry challenges.
With this release, the OpenMP API will move immediately forward to the next release to bring even more usable parallelism to everyone.
 – Michael Wong, CEO OpenMP ARB.


Hunger Games themed semi-iterated prisoner’s dilemma tournament

With all the talk surrounding it, crowdsourcing science might seem like a new concept and it might be true for citizen science efforts, but it is definitely an old trick to source your research to other researchers. In fact, evolutionary game theory was born (or at least popularized) by one such crowdsourcing exercise; in 1980, Robert Axelrod wanted to find out the best strategy for iterated prisoner’s dilemma and reached out to prominent researchers for strategy submissions to around-robin tournmanet. Tit-for-tat was the winning strategy, but the real victor was Axelrod. His 1981 paper with Hamilton analyzing the result went on to become a standard reference in applications of game theory to social questions (at least outside of economics), agent-based modeling, and — of course — evolutionary game theory. Of Axelrod’s sizeable 47,222 (at time of writing) citations, almost half (23,370) come from this single paper. The tradition of tournaments continues among researchers, I’ve even discussed an imitation tournament on imitation previously.

The cynical moral of the tale: if you want to be noticed then run a game theory tournament. The folks at Brilliant.org— a website offering weekly olympiad-style challange problems in math and physics — took this message to heart, coupled it to the tried-and-true marketing technique of linking to a popular movie/book franchise, and decided to run a Hunger Games themed semi-iterated Prisoner’s dillema tournament. Submit a quick explanation of your strategy and Python script to play the game, and you could be one of the 5 winners of the $1,000 grand prize. Hooray! The submission deadline is August 18th, 2013 and all you need is a Brilliant account and it seems that these are free. If you are a reader of TheEGG blog then I recommend submitting a strategy, and discussing it in the comments (either before or after the deadline); I am interested to see what you come up with.

I will present the rules in m


Official feedback on OpenGL 4.4 thread

 SIGGRAPH – Anaheim, CA – The Khronos™ Group today announced the immediate release of the OpenGL® 4.4 specification,bringing the very latest graphics functionality to the most advanced and widely adopted cross-platform 2D and 3D graphics API (application programming interface). OpenGL 4.4 unlocks capabilities of today’s leading-edge graphics hardware while maintaining full backwards compatibility, enabling applications to incrementally use new features while portably accessing state-of-the-art graphics processing units (GPUs) across diverse operating systems and platforms. Also, OpenGL 4.4 defines new functionality to streamline the porting of applications and titles from other platforms and APIs. The full specification and reference materials are available for immediate download at http://www.opengl.org/registry.

In addition to the OpenGL 4.4 specification, the OpenGL ARB (Architecture Review Board) Working Group at Khronos has created the first set of formal OpenGL conformance tests since OpenGL 2.0. Khronos will offer certification of drivers from version 3.3, and full certification is mandatory for OpenGL 4.4 and onwards. This will help reduce differences between multiple vendors’ OpenGL drivers, resulting in enhanced portability for developers.

New functionality in the OpenGL 4.4 specification includes:

Buffer Placement Control (GL_ARB_buffer_storage)
Significantly enhances memory flexibility and efficiency through explicit control over the position of buffers in the graphics and system memory, together with cache behavior control – including the ability of the CPU to map a buffer for direct use by a GPU.

Efficient Asynchronous Queries
(GL_ARB_query_buffer_object)
Buffer objects can be the direct target of a query to avoid the CPU waiting for the result and stalling the graphics pipeline. This provides significantly boosted performance for applications that intend to subsequently use the results of queries on the GPU, such as dynamic quality reduction strategies based on performance metrics.

Shader Variable Layout (GL_ARB_enhanced_layouts)
Detailed control over placement of shader interface variables, including the ability to pack vectors efficiently with scalar types. Includes full control over variable layout inside uniform blocks and enables shaders to specify transform feedback variables and buffer layout.

Efficient Multiple Object Binding (GL_ARB_multi_bind)
New commands which enable an application to bind or unbind sets of objects with one API call instead of separate commands for each bind operation, amortizing the function call, name space lookup, and potential locking overhead. The core rendering loop of many graphics applications frequently bind different sets of textures, samplers, images, vertex buffers, and uniform buffers and so this can significantly reduce CPU overhead and improve performance.

Streamlined Porting of Direct3D applications

A number of core functions contribute to easier porting of applications and games written in Direct3D including GL_ARB_buffer_storage for buffer placement control, GL_ARB_vertex_type_10f_11f_11f_rev which creates a vertex data type that packs three components in a 32 bit value that provides a performance improvement for lower precision vertices and is a format used by Direct3D, and GL_ARB_texture_mirror_clamp_to_edge that provides a texture clamping mode also used by Direct3D.Extensions released alongside the OpenGL 4.4 specification include:

Bindless Texture Extension (GL_ARB_bindless_texture)
Shaders can now access an effectively unlimited number of texture and image resources directly by virtual addresses. This bindless texture approach avoids the application overhead due to explicitly binding a small window of accessible textures. Ray tracing and global illumination algorithms are faster and simpler with unfettered access to a virtual world’s entire texture set.

Sparse Texture Extension (GL_ARB_sparse_texture)
Enables handling of huge textures that are much larger than the GPUs physical memory by allowing an application to select which regions of the texture are resident for ‘mega-texture’ algorithms and very large data-set visualizations.

OpenGL BOF at SIGGRAPH, Anaheim, CA July 24th 2013
There is an OpenGL BOF “Birds of a Feather” Meeting on Wednesday July 24th at 7-8PM at the Hilton Anaheim, California Ballroom A & B, where attendees are invited to meet OpenGL implementers and developers and learn more about the new OpenGL 4.4 specification.


Hacking with a Hacker

What is it like to hack with one of the original hackers? It is certainly much different than what Appears to be the modern rendition of hacking. My experience was not getting really drunk with tons of junk food. It was not working on “beautiful” designs or “authentic” typography. It was not so much about sharing with the world as it was sharing with your peers. It had a very different feel to it than the “hacker culture” Promoted by some of the top technical Silicon Valley companies. It felt more “at home”, less dreamy, and more memorable.

I meet with Bill Gosper every so Often; I had the pleasure of giving him a tour of Facebook when I worked there. (He was so surprised that they had Coke in the glass bottles there, just like the old days.)

He is still very much a hacker, a thinker, a tinkerer, and a wonderer. Every time I meet up with him, he has a new puzzle for me, or someone around him, to solve, whether it’s really clever compass constructions, circle packing, block building, Game of Life automata solving, or even something more tangible like a Buttonhole homemade trap (which was affixed to my shirt for no less than two weeks!). He is also the bearer of interesting items, such as a belt buckle he gave me roomates depicts, in aluminum, a particular circle loose packing.
Gosper succeeding in tricking me with the Buttonhole Trap
When we meet up, all we do is hack. Along with him and one of his talented young students, we all work on something. Anything interesting, really. Last time we met up, we resurrected an old Lisp machine and did some software archeology. I brought over some of the manuals I own, like the great Chinual, and he brought over a dusty old 1U rackmount Alpha machine with OpenGenera installed. After passing a combination of Hurdles, such as that the keyboard was not interfacing with the computer Correctly, we finally got it to boot up. Now, I got to see with my own eyes, a time capsule containing a lot of Bill’s work from the 70s, 80s, and 90s, roomates could only be commanded and Examined through Zmacs dired and Symbolics Common Lisp. Our next goal was to get Symbolics Macsyma fired up on the old machine.

There was trouble with starting it up. License issues were one problem, finding and loading all of the files were compiled another. Running applications on a Lisp machine is very different than what we do today on modern machines, Windows or UNIX. There’s no. Exe file to click, or. App bundle to start up, or even a single. / File to execute. Usually it’s a collection of compiled “fast loading” or “fasl” files that get loaded side-by-side with the operating system. The application, in essence, Becomes a part of the OS.

In hacker tradition, we were Able to bypass the license issues by modifying the binary directly in Lisp. Fortunately, such as Lisp makes things easy disassembly. But how do we load the damn thing? Bill frustratingly muttered, “It’s been at least 20 years since I’ve done it. I just do not remember. “I, being an owner of MacIvory Symbolics Lisp machines, fortunately did remember how to load programs. “Bill, how about LOAD SYSTEM Macsyma?” He typed it into the native Lisp “Listener 2” window (we kept “Listener 1” for debugging), sometimes making a few typing mistakes, but finally succeeding, and then we saw the stream of files loading. We all Shouted in joy that progress was being made. I recall Bill was especially astounded at how fast everything was loading. This was on a fast Alpha machine with gobs of memory. It must have been much slower on the old 3600s they used back in the day.
The Lisp Machine Manual, or Chinual
It was all done after a few minutes, and Macsyma was loaded. To me, this was a sort of holy grail. I personally have Macsyma for Windows (which he uses in a VirtualBox machine on his 17 “MacBook), and I’ve definitely used Maxima. But Macsyma is something I’ve never seen. It was something that seems to have disappeared with history, something I have not been Able to find a copy of in the last decade.

Bill said, “let’s see if it works.” And he typed 1 +1; in, and sure enough, the result was 2. He saw I was bubbling with excitement and asked me if I’d like to try anything. “I’d love to,” and he handed the keyboard over to me and I typed in my canonical computer algebra test: integrate (sqrt (tan (x)), x);, roomates computes the indefinite integral
—- √ ∫ tanθ dθ
Out came the four-term typeset result of logarithms and arctangents, plus a fifth term I’d never seen before. “I’ve never seen any computer algebra system add that fifth term,” I said, “but it does not look incorrect.” The fifth term was a floored expression, Whose Increased value with the period of the function preceding it. “Let’s plot it,” Bill said. He plotted it using Macsyma’s menu interface, and it was what appeared to be an increasing, non-periodic function. I think we determined it was properly handled Because Macsyma branch cuts, with other systems have been known to be unorthodox about. It definitely had a pragmatic feel to it.

Now, Bill wanted to show us some interesting things; however all of Bill’s recent work Macsyma was on his laptop. How do we connect this ancient to a modern Macintosh hardware? We needed to get the machine onto the network, and networking with old machines is not my forte.

Fortunately, Stephen Jones, a friend of Bill’s and seemingly an expert at a rare combination of technical tasks, showed up. He Was able to do things that Neither Bill nor I could do on such an old machine. In only a few moments, he Was able to get Bill’s Mac talking to the Alpha, roomates shared a portion of its file system with Genera. “Will there be enough space on the Alpha for Macsyma my files?” Bill asked Stephen. “Of course, there’s ton’s of space.” We finally got Bill’s recent work transferred onto the machine.
Bill hacking in Macsyma in OpenGenera (Image courtesy of Stephen M. Jones)
We spent the rest of the night hacking on math. He Demonstrated to us what it was like to do a real mathematician’s work at the machine. He debuted some of his recent work. He went though a long chain of reasoning, showing us the line-after-line in Macsyma, number theoretic amazing to do things I’ve never seen before.

I did ask Bill why he does not publish more often. His previous publications have been landmarks: his algorithm for hypergeometric series and his summation algorithm for playing the Game of Life at light speed. He RESPONDED, “when there’s something interesting to publish, it’ll be published.” He seemed to have a sort of disdain for “salami science”, where scientific and mathematical papers present the thinnest possible “slice” or result possible.

Bill is certainly a man that thinks in a different way than most of us do. He is still hacking at mathematics, and still as impressive as before. I’m very fortunate to have met him, and I was absolutely delighted to see that even at 70 years old, his mind is still as sharp as can be, and it’s still being used to do interesting, Gosper-like mathematics.

And you would not believe it. We all were ready to head home at around 9 PM.


Variabel di Shell Script

In the previous issue we’ve become acquainted with a shell script and managed to create a very simple shell script. If we look, we did not perform any processing on the shell script. We only show a message on the screen and execute commands on the shell through shell script. What if we want to make the program more interactive shell script?

As with any programming language, shell script also serves to recognize the variables that can hold information temporarily for a variety of purposes, for example to compute or determine the output results. You may make as much as possible or use a variable in your shell script. Name the variables are independent, large and small letters should, but make it easier to remember, make it a habit to create standard rules in the manufacture of the variable name. At this writing, all variables will be written in lowercase.

Variables can be divided into two types, namely environment variables and user variables. Environment variables are variables that have been previously determined as a part of the shell used (bash). By default, the name of this variable using all capital letters. Example of this is the $ USER variable that will contain the user name you are currently using, $ HOME home directory which contains the address of the user that is used, and so forth. To display the entire value of an existing environment variable, you can use the set command in the terminal (Figure 1). User variable is the variable name specified by the user themselves and not by the shell used.

Variables can be accessed by using the dollar sign ($) before the variable name, for example, we have a variable named “my name”, to access the value stored in the variable, we use the $ my name. To give value to a variable, we use the sign “=” is immediately followed by the value we give without any spaces, for example my name = Willy. What if the value that we want to give is a sentence? Use double quotes as the opening and closing value of a variable, such as my name = “Willy Sudiarto Raharjo”. For example, see listing 1 and try running on your computer. Seen that the value of the variable Sudiarto be regarded as a command and not part of the variable because it is not enclosed in double quotation marks. Please be careful in giving a value to a variable.

You can combine environment variables and user variables in a shell script is the same, as in listing 2. What if want to write a message using the $ character, such as “It cost me $ 15”? If we are not careful, it could be a shell script would be wrong to interpret the information that we provide and try to take the value of variable 1 (which will not contain any) and display it as a “price 5”. To fix this, use the escape character to indicate that the next character will be recognized as a regular character and not as a substitute for a variable, which marks the backslash “” as in listing 3.

One character that needs more attention is the backtick character “` “(position number 1 on the left side your keyboard) because this character has a special function in shell programming, which is able to accommodate the output of a shell command in a variable. As an example, we will hold the result of the date command into a variable date and display its contents using the echo command as in listing 4.

To be able to receive input from the user and store it into a variable, we can use the read function is followed by the name of the variable that we want to use to store the values ​​as in Example 5 listings.


It’s a Mad, Mad, Mad, Mad World: Scoping in CoffeeScript and JavaScript

CoffeeScript, as many people know, is a transpile-to-JavaScript language.1 For the most part, it does not introduce major changes in semantics. For example, this:

-> 'Hello, world'

Transpiles directly to:

function () { return 'Hello, world'; }

This is convenient syntactic sugar, and by removing what some folks call the “syntactic vinegar” of extraneous symbols, it encourages the use of constructs that would otherwise make the code noisy and obscure the important meaning. The vast majority of features introduced by CoffeeScript are of this nature: They introduce local changes that transpile directly to JavaScript.2

CoffeeScript also introduces features that don’t exist in JavaScript, such as destructuring assignment and comprehensions. In each case, the features compile directly to JavaScript without introducing changes elsewhere in the program. And since they don’t look like existing JavaScript features, little confusion is created.

equals doesn’t equal equals

One CoffeeScript feature does introduce confusion, and the more you know JavaScript the more confusion it introduces. This is the behaviour of the assignment operator, the lowly (and prevalent!) equals sign:

foo = 'bar'

Although it looks almost identical to assignment in JavaScript:

foo = 'bar';

It has different semantics. That’s confusing. Oh wait, it’s worse than that: Sometimes it has different semantics. Sometimes it doesn’t.

So what’s the deal with that?

Well, let’s review the wonderful world of JavaScript. We’ll pretend we’re in a browser application, and we write:

foo = 'bar';

What does this mean? Well, it depends: If this is in the top level of a file, and not inside of a function, then foo is a global variable. In JavaScript, global means global across all files, so you are now writing code that is coupled with every other file in your application or any vendored code you are loading.

But what if it’s inside a function?

function fiddleSticks (bar) {
  foo = bar;
  // ...
}

For another example, many people enclose file code in an Immediately Invoked Function Expression (“IIFE”) like this:

;(function () {
  foo = 'bar'
  // more code...
})();

What do foo = 'bar'; or foo = bar; mean in these cases? Well, it depends as we say. It depends on whether foo is declared somewhere else in the same scope. For example:

function fiddleSticks (bar) {
  var foo;
  foo = bar;
  // ...
}

Or:

function fiddleSticks (bar) {
  foo = bar;
  // ...
  var foo = batzIndaBelfrie;
  // ...
} 

Or even:

function fiddleSticks (bar) {
  foo = bar;
  // ...
  function foo () {
    // ...
  }
  // ...
}

Because of something called hoisting,3 these all mean the same this: foo is local to function fiddleSticks, and therefore it is NOT global and ISN’T magically coupled to every other file loaded whether written by yourself or someone else.

nested scope

JavaScript permits scope nesting. If you write this:

function foo () {
  var bar = 1;
  var bar = 2;
  return bar;
}

Then bar will be 2. Declaring bar twice makes no difference, since both declarations are in the same scope. However, if you nest functions, you can nest scopes:

function foo () {
  var bar = 1;
  function foofoo () {
    var bar = 2;
  }
  return bar;
}

Now function foo will return 1 because the second declaration of bar is inside a nested function, and therefore inside a nested scope, and therefore it’s a completely different variable that happens to share the same name. This is called shadowing: The variablebar inside foofoo shadows the variable bar inside foo.

javascript failure modes

Now over time people have discovered that global variables are generally a very bad idea, and accidental global variables doubly so. Here’s an example of why:

function row (numberOfCells) {
  var str = '';
  for (i = 0; i < numberOfCells; ++i) {
    str = str + '<td></td>';
  }
  return '<tr>' + str + '</tr>';
}

function table (numberOfRows, numberOfColumns) {
  var str = '';
  for (i = 0; i < numberOfRows; ++i) {
    str = str + row(numberOfColumns);
  }
  return '<table>' + str + '</table>';
}

Let’s try it:

table(3, 3)
  //=> "<table><tr><td></td><td></td><td></td></tr></table>"

We only get one row, because the variable i in the function row is global, and so is the variable i in the function table, so they’re the exact same global variable. Therefore, after counting out three columns, i is 3 and the for loop in table finishes. Oops!

And this is especially bad because the two functions could be anywhere in the code. If you accidentally use a global variable and call a function elsewhere that accidentally uses the same global variable, pfft, you have a bug. This is nasty because there’s this weird action-at-a-distance where a bug in one file reaches out and breaks some code in another file.

Now, this isn’t a bug in JavaScript the language, just a feature that permits the creation of very nasty bugs. So I call it a failure mode, not a language bug.

coffeescript to the rescue

CoffeeScript addresses this failure mode in two ways. First, all variables are local to functions. If you wish to do something in the global environment, you must do it explicitly. So in JavaScript:

UserModel = Backbone.Model.extend({ ... });
var user = new UserModel(...);

While in CoffeeScript:

window.UserModel = window.Backbone.Model.extend({ ... })
user = new window.UserModel(...)

Likewise, CoffeeScript bakes the IIFE enclosing every file in by default. So instead of:

;(function () {
  // ...
})();

You can just write your code.4

The net result is that it is almost impossible to replicate the JavaScript failure mode of creating or clobbering a global variable by accident. That is a benefit.

what would coffeescript do?

This sounds great, but CoffeeScript can be surprising to JavaScript programmers. Let’s revisit our table function. First, we’ll fix it:

function row (numberOfCells) {
  var i,
      str = '';
  for (i = 0; i < numberOfCells; ++i) {
    str = str + '<td></td>';
  }
  return '<tr>' + str + '</tr>';
}

function table (numberOfRows, numberOfColumns) {
  var i,
      str = '';
  for (i = 0; i < numberOfRows; ++i) {
    str = str + row(numberOfColumns);
  }
  return '<table>' + str + '</table>';
}

table(3, 3)
  //=> "<table><tr><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td></tr></table>"

Good! Now suppose we notice that no function calls row other than table. Although there is a slightly more “performant” way to do this, we decide that the clearest and simplest way to indicate this relationship is to nest row inside table Pascal-style:

function table (numberOfRows, numberOfColumns) {
  var i,
      str = '';
  for (i = 0; i < numberOfRows; ++i) {
    str = str + row(numberOfColumns);
  }
  return '<table>' + str + '</table>';

  function row (numberOfCells) {
    var i,
        str = '';
    for (i = 0; i < numberOfCells; ++i) {
      str = str + '<td></td>';
    }
    return '<tr>' + str + '</tr>';
  }
}

It still works like a charm, because the i in row shadows the i in table, so there’s no conflict. Okay. Now how does it work in CoffeeScript?

Here’s one possible translation to CoffeeScript:

table = (numberOfRows, numberOfColumns) ->
  row = (numberOfCells) ->
    str = ""
    i = 0
    while i < numberOfCells
      str = str + "<td></td>"
      ++i
    "<tr>" + str + "</tr>"
  str = ""
  i = 0
  while i < numberOfRows
    str = str + row(numberOfColumns)
    ++i
  return "<table>" + str + "</table>"

table(3,3)
  #=> "<table><tr><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td></tr></table>"

It works just fine. Here’s another:

table = (numberOfRows, numberOfColumns) ->
  str = ""
  i = 0
  row = (numberOfCells) ->
    str = ""
    i = 0
    while i < numberOfCells
      str = str + "<td></td>"
      ++i
    "<tr>" + str + "</tr>"
  str = ""
  i = 0
  while i < numberOfRows
    str = str + row(numberOfColumns)
    ++i
  return "<table>" + str + "</table>"

table(3,3)
  #=> "<table><tr><td></td><td></td><td></td></tr></table>"

Broken! And a third:

str = ""
i = 0
table = (numberOfRows, numberOfColumns) ->
  row = (numberOfCells) ->
    str = ""
    i = 0
    while i < numberOfCells
      str = str + "<td></td>"
      ++i
    "<tr>" + str + "</tr>"
  str = ""
  i = 0
  while i < numberOfRows
    str = str + row(numberOfColumns)
    ++i
  return "<table>" + str + "</table>"

table(3,3)
  #=> "<table><tr><td></td><td></td><td></td></tr></table>"

Also broken! Although the three examples look similar, the first gives us what we expect but the second and third do not. What gives?

Well, CoffeeScript doesn’t allow us to “declare” that variables are local with var. They’re always local. But local to what? In CoffeeScript, they’re local to the function that either declares the variable as a parameter or that contains the first assignment to the variable.5 So in our first example, reading from the top, the first use of str and i is inside the row function, so CoffeeScript makes them local to row.

A little later on, the code makes an assignment to i and str within the table function. This scope happens to enclose row’s scope, but it is different so it can’t share the strand i variables. CoffeeScript thus makes the i and str in table variables local totable. As a result, the i and str in row end up shadowing the i and str in table.

The second example is different. The first i encountered by CoffeeScript is in table, so CoffeeScript makes it local to table as we’d expect. The second i is local to row. But since row in enclosed by table, it’s possible to make that i refer to the i already defined, and thus CoffeeScript does not shadow the variable. The i inside row is the same variable as the i inside table.

In the third example, i (and str) are declared outside of both table and row, and thus again they all end up being the same variable with no shadowing.

Now, CoffeeScript could scan an entire function before deciding what variables belong where, but it doesn’t. That simplifies things, because you don’t have to worry about a variable being declared later that affects your code. Everything you need to understand is in the same file and above your code.

In many cases, it also allows you to manipulate whether a variable is shadowed or not by carefully controlling the order of assignments. That’s good, right?

all those against the bill, say “nay nay!”

Detractors of this behaviour say this is not good. When JavaScript is written using var, the meaning of a function is not changed by what is written elsewhere in the file before the code in question. Although you can use this feature to control shadowing by deliberately ordering your code to get the desired result, a simple refactoring can break what you’ve already written.

For example, if you write:

table = (numberOfRows, numberOfColumns) ->
  row = (numberOfCells) ->
    str = ""
    i = 0
    while i < numberOfCells
      str = str + "<td></td>"
      ++i
    "<tr>" + str + "</tr>"
  str = ""
  i = 0
  while i < numberOfRows
    str = str + row(numberOfColumns)
    ++i
  return "<table>" + str + "</table>"

table(3,3)
  #=> "<table><tr><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td></tr><tr><td></td><td></td><td></td></tr></table>"

All will be well, until you are debugging late one night, and you add:

console.log('Hello!') for i in [1..5]

table = (numberOfRows, numberOfColumns) ->
  row = (numberOfCells) ->
    str = ""
    i = 0
    while i < numberOfCells
      str = str + "<td></td>"
      ++i
    "<tr>" + str + "</tr>"
  str = ""
  i = 0
  while i < numberOfRows
    str = str + row(numberOfColumns)
    ++i
  return "<table>" + str + "</table>"

table(3,3)
  #=> "table><tr><td></td><td></td><td></td></tr></table>"

This breaks your code because the i you used at the top “captures” the other variables so they are now all the same thing. To someone used to JavaScript, this is a Very Bad Thing™. When you write this in JavaScript:

function row (numberOfCells) {
  var i,
      str = '';
  for (i = 0; i < numberOfCells; ++i) {
    str = str + '<td></td>';
  }
  return '<tr>' + str + '</tr>';
}

It will always mean the same thing no matter where it is in a file, and no matter what comes before it or after it. There is no spooky “action-at-a-distance” where code somewhere else changes what this code means. Whereas in CoffeeScript, you don’t know whether the iin row is local to row or not without scanning the code that comes before it in the same or enclosing scopes.

coffeescript’s failure mode

In this case, CoffeeScript has a failure mode: The meaning of a function seems to be changed by altering its position within a file or (in what amounts to the same thing) by altering code that appears before it in a file in the same or enclosing scopes. In contrast, JavaScript’s var declaration never exhibits this failure mode. JavaScript has a different action-at-a-distance failure mode, where neglecting var causes action at a much further distance: The meaning of code can be affected by code written in an entirely different file.

Mind you, the result of calling our row function is not affected by declaring an i in an enclosing scope. Our function always did what it was expected to do and always will. Although you and I know why the change breaks the table function is that row now uses an enclosed variable, imagine that we were writing unit tests. All of our tests for row would continue to pass, it’s the tests for table that break. So in an evidence-based programming sense, when we maintain the habit of always initializing variables we expect to use locally, changing code outside of those functions only changes the evidence that the enclosing code produces.

So one way to look at this is that row is fine, but moving i around changes the meaning of the code where you move i. And why wouldn’t you expect making changes to tableto change its meaning?

so which way to the asylum?

If you ask around, you can find people who dislike JavaScript’s behaviour, and others who dislike CoffeeScript’s behaviour. Accidentally getting global variables when you neglectvar is brutal, and action-at-a-distance affecting the meaning of a function (even if it is always within the same file) flies against everything we have learned about the importance of writing small chunks of code that completely encapsulate their behaviour.

Of course, programmers tend to internalize the languages they learn to use. If you write a lot of JavaScript, you habitually use var and may have tools that slap your wrist when you don’t. You’re bewildered by all this talk of action-at-a-distance. It will seems to you to be one of those rookie mistake problems that quickly goes away and is not a practical concern.

Likewise, if you write twenty thousand lines of CoffeeScript, you may never be bitten by its first-use-is-a-declaration behaviour. You may be in the habit of using variable names likeiRow and iColumn out of habit. You may find that your files never get so large and your functions so deeply nested that a “capture” problem takes longer than three seconds to diagnose and fix.

It’s a bit of a cop-out, but I suggest that this issue resembles the debate over strong, manifest typing vs. dynamic typing. In theory, one is vastly preferable to the other. But in practice, large stable codebases are written with both kinds of languages, and programmers seem to adjust to overcome the failure modes of their tools unconsciously while harvesting the benefits that each language provides.


  1. Yes, “transpile” is a real word, or at least, a real piece of jargon. It’s a contraction of “transcompiler,” which is a compiler that translates one language to another language at a similar level of abstraction. There’s room for debate over what constitutes a “similar level of abstraction.” https://en.wikipedia.org/wiki/Source-to-source_compiler

  2. There are other possibilities: You could write a Tail-Call Optimized language that transpiles to JavaScript, however its changes wouldn’t always be local: Some function calls would be rewritten substantially to use trampolining. Or adding continuations to a language might cause everything to be rewritten in continuation-passing style.

  3. Scanning all of the code first is called “hoisting,” in part because some declarations nested in blocks are “hoisted” up to the level of the function, and all declarations are “hoisted” to the top of the function. This is a source of confusion for some programmers, but it isn’t germane to this essay.

  4. If you don’t want the file enclosed in an IIFE, you can compile your CoffeeScript with the--bare command-line switch.