Hacker News Personal Blogs 2015 | All Blog Posts

Play2: Switching from specs2 to ScalaTest

Home on The Log Book of Manuel Kießling — Thu, 31 Dec 2015 17:13:00 GMT

The following applies to Play 2.4 projects for Scala 2.11. When setting up a Play2 Scala project, two test case files are created automatically: test/ApplicationSpec.scala and test/IntegrationSpec.scala. The testcases therein are based on specs2. If you prefer to write your test cases using ScalaTest (I certainly do), then you need to proceed as follow: Change the libraryDependencies in file build.sbt from: libraryDependencies ++= Seq( jdbc, cache, ws, specs2 % Test )

Ten noteworthy books I read in 2015

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Thu, 31 Dec 2015 15:36:57 GMT

Since getting a Kindle last Christmas, I have started reading even more. This is my personal list of noteworthy books I read in 2015. In no particular order, we have:

Seveneves, by Neal Stephenson: A truly epic tale of the near-extinction and re-birth of humanity. In some ways, this book even rivals the love I feel towards Stephenson’s Cryptonomicon. Read this if you are even slightly interested in science or space.
The Psychology of Invention in the Mathematical Field, by Jacques Hadamard: I was overjoyed to find this in the bargain bin of a bookshop. Hadamard studies nothing less than the way mathematicians invent (or discover) mathematics. The result is an engaging work with (unfortunately) vague answers that are not generally applicable. Still, it rekindled my interest in these topics and I look forward to reading more formalized works from this area.
Effective Modern C++, by Scott Meyers: Meyers continues to amaze me with his in-depth knowledge of this most peculiar programming language (…as majestic as troops with banners…). If you want to do serious work with C++11 or C++14, read this book. No excuses.
Embassytown, by China Miéville: A captivating story about humanity’s contact with a truly alien race and their unique language. If you ever wondered about what a species would live like if the (discredited) Sapir-Whorf hypothesis was fully true, you will enjoy this book as much as I did.
Parable of the Sower, by Octavia E. Butler: Set in a failing world that is not much unlike our own, Butler describes the tale of a young woman and her struggle for survival. In contrast to many other books describing an apocalyptic event, this one really hit a spot because the reader can feel the world crumble while reading—and the fall is not preceded by loud bangs but rather by quiet whispers.
The Opposite of Loneliness: Essays and Stories, by Marina Keegan: Published posthumously after the untimely death of this young writer, this is a collection of her work. At times, it was painful to read with the knowledge that the spark of a great writer that is easily glimpsed in the book would never mature. The stories speak of a hunger for life, of possibilities, of uncertainty, and of fear. If you read only one essay, read Even Artichokes have Doubts.
Goddess of the Market, by Jennifer Burns: After reading this biography of Ayn Rand, I may finally re-read her books with more sympathy and more appreciation. Read this if you think that Atlas Shrugged is “not a book to be tossed away lightly—it should be hurled with great forced”. At the very least, you may understand the thought patterns of political conservatives and libertarians better.
After the Fall, Before the Fall, During the Fall, by Nancy Kress: In my ongoing quest to read more books written by women, I had already encountered the writings of Nancy Kress. Although this was also a book about the apocalypse, I enjoyed the way Kress intertwined multiple perspectives about The Fall. Although the resolution of the novel is slightly campy, this is still a gripping tale.
Muse of Fire, by Dan Simmons: Shakespeare in space—what’s not to love? After Simmons wrote himself into my heart with his extremely intelligent Hyperion Cantos and the epic Ilium/Olympos books, Muse of Fire was another formidable book in the genre of “literary science fiction”.
Visualization Analysis and Design, by Tamara Munzner: A must-read in my profession. Munzner gives us a flexible framework that helps us think in a more structured manner about doing visualization.

According to goodreads, I read 63 books this year, with a total of 19131 pages. The shortest book I read had 60 pages, while the longest had 880. The average number of pages was 314. I initially wanted to read 25 books this years but got somewhat carried away.

Good reading for 2016!

Gene Regulatory Networks

Playground — Thu, 31 Dec 2015 02:00:00 GMT

Here goes the documentation of the work I did during my summer internship. Yeah, I know, this is coming quite, quite late, but hey! Better late than never! All the coding was done in R (check out the repo). There will also be another post which will talk about the technical (basically, R code) details.

Private data for JS classes with WeakMap

Qubyte Codes — Wed, 30 Dec 2015 16:35:00 GMT

Private data has always been awkward in JavaScript. It's particularly difficult when it comes to constructors, and with ES2015 recently published, classes too. Let's say we have an example class, exported by an ES2015 module:

export default class Example {
  constructor() {
    this._privateDatum = Math.random();
  }

  log() {
    console.log(this._privateDatum);
  }
}

In the constructor, a field with some private data, _privateDatum is appended (the value is a placeholder for illustration). The initial underscore in the name is a common convention and is meant to tell developers using the class that they shouldn't touch or look at that field. Why should this be private? Private stuff is subject to change without your users needing to know about it. This field could be renamed or go away completely if you refactor, without affecting the public API. So what's the problem?

You can't trust your users!

This isn't meant as an insult. Your users are cunning, and if they can solve a problem without filing an issue or raising a pull request, they probably will. They have deadlines after all... If your class gets very popular, it becomes inevitable that someone is going to use your private-by-convention field to hack together a solution to a problem they're having, and you'll break their code when you change it. Changes to your public API should be clearly indicated by changes to the version number and updated documentation. The inner workings of your code on the other hand, including private data, are subject to dramatic change at any time.

The solution is to hide the private data, removing the temptation. You can do this using a closure. Consider:

export default class Example {
  constructor() {
    const privateDatum = Math.random();

    this.log = function () {
      console.log(privateDatum);
    }
  }
}

In this example, the private data is now assigned to a variable in the constructor. Since the variable is not returned, nothing outside the constructor will have access to it. The pain now is that the log method has to be attached to the instance inside the constructor, so that it can have access to the variable. It's a real shame to lose the nice method syntax. It also make an individual log method for each instance, which means objects will each use more memory.

This is where WeakMap comes in. An instance of WeakMap has keys which are objects of some kind, and values which can be whatever you like. WeakMap instances are especially good, since if they are the last thing to hold a reference to an object (as a key), then the JS engine is allowed to garbage collect it. This means the risk of memory leaks is lessened. You could simulate most aspects of WeakMap using existing structures like arrays, but that would always result in a memory leak, since the garbage collector thinks those objects are in use and cannot clear them up. The final example below shows what this looks like:

const privateDatum = new WeakMap();

export default class Example {
  constructor() {
    privateDatum.set(this, Math.random());
  }

  log() {
    console.log(privateDatum.get(this));
  }
}

The keys of privateDatum are the instances of the example class. If nothing else holds a reference to an instance of the example class, the garbage collector doesn't count the reference in privateDatum and can clear it up! Since the instances are keys, this can be used in any method to access the private data. The privateDatum variable hidden by the module, so the user will have no access to it!

This approach can be used with constructor functions and methods appended to the prototype too. The following constructor produces objects with similar behaviour to those produced by the class in the previous example:

const privateDatum = new WeakMap();

export default function Example() {
  privateDatum.set(this, Math.random());
}

Example.prototype.log = function () {
  console.log(privateDatum.get(this));
};

The good news is that WeakMap is one of the most well supported features of ES2015. With the exception of IE Mobile and Opera Mobile, all current versions of major browsers support the functionality in this post. See the compatibility table. If you're using a maintained version of Node, you're good to go!

Addendum

It's still possible to gain access to private data stored in a WeakMap by patching WeakMap.prototype.set or WeakMap.prototype.get. This should absolutely never be done! Along with the usual reasons to not modify the prototype of a built in constructor, modifying WeakMap risks undoing the whole reason for using it in the first place. By monitoring objects used as the keys of a WeakMap, references can be created and the garbage collector may not be able to clean up after you. That said, patching can be done. If you want to avoid that risk, you can Object.freeze both WeakMap and WeakMap.prototype before any other code runs.

Find in \$PATH with type and which

xenodium.com @alvaro — Wed, 30 Dec 2015 02:00:00 GMT

I typically use which to figure out the first binary found in $PATH:

which -a emacsclient

/Users/user/homebrew/bin/emacsclient
/usr/bin/emacsclient

I always forget about type though:

type -a emacsclient

emacsclient is /Users/user/homebrew/bin/emacsclient
emacsclient is /usr/bin/emacsclient

npm basics

xenodium.com @alvaro — Wed, 30 Dec 2015 02:00:00 GMT

Global vs local package installation location

{prefix}/lib/node_modules

path/to/project/node_modules

View npm config

npm config list

; cli configs
user-agent = "npm/2.14.2 node/v4.0.0 darwin x64"

; node bin location = /Users/user/.nvm/versions/node/v4.0.0/bin/node
; cwd = /Users/user/stuff/active/blog
; HOME = /Users/user
; 'npm config ls -l' to show all defaults.

Get config value

npm config get prefix

/Users/user/.nvm/versions/node/v4.0.0

Set config value

npm config set prefix=$HOME/some/location

Install package globally

node install --global <package-name>

node install -g <package-name>

List global packages

npm list --global

You can also use –depth=0 to make less verbose.

/Users/user/.nvm/versions/node/v4.0.0/lib
├─┬ babel-eslint@4.1.3
│ ├── acorn-to-esprima@1.0.4
│ ├─┬ babel-core@5.8.25
│ │ ├── babel-plugin-constant-folding@1.0.1
│ │ ├── babel-plugin-dead-code-elimination@1.0.2
...

Install local package

npm install <package-name> --save

–save will add <package-name> dependency to your package.json.

package.json

See using a package.json.

Uninstall package

npm uninstall <package-name>

Install package at version

npm install <package-name>@1.7.0

Search packages

npm search linter

Online documentation

Online documentation is great so far. More at docs.npmjs.com.

Crowd Surfing

Justin Garrison's Homepage — Wed, 30 Dec 2015 02:00:00 GMT

An injury from 1999 that still affects me today.

iPhone vs. Android vs. webOS: A Counterpoint

Justin Garrison's Homepage — Wed, 30 Dec 2015 02:00:00 GMT

Earlier this week, we compared the iPhone and Android in a knockdown showdown of features that matter to us.

2015 in review

Jason Eckert's Website and Blog — Wed, 30 Dec 2015 02:00:00 GMT

2015 was an interesting year, and one that I really enjoyed!

I spent a lot of time getting/restoring/playing the last few arcade machines that I’ve wanted in my arcade collection - I just love the retro arcade hobby! As a result, I won’t be getting or restoring any more in 2016 at all (I have a few arcades that I want to work on to upgrade, but that’s about it). I’ll be spending most of my time playing them!

My top 10 Postgres features and tips for 2016

CRAIG KERSTIENS — Tue, 29 Dec 2015 22:55:56 GMT

I find during the holiday season many pick up new books, learn a new language, or brush up on some other skill in general. Here’s my contribution to hopefully giving you a few new things to learn about Postgres and ideally utilize in the new year. It’s not in a top 10 list as much as 10 tips and tricks you should be aware of as when you need them they become incredibly handy. But, first a shameless plug if you find any of the following helpful, consider subscribing to Postgres weekly a weekly newsletter with interesting Postgres content.

1. CTEs - Common Table Expressions

CTEs allow you to do crazy awesome things like recursive queries but even the most simple form of them I don’t go a day without using. Think of a CTE or commonly known as with clause as a view inside the time that query is running. This lets you more easily create readable query. Any query that’s constructed that’s even 100 lines long, but with 4-5 CTEs is undoubtedly going to be easier for someone new to come in and understand than a 20 line query that does the same thing. A few people like writing SQL, but no one likes reading someone else’s so do them a favor and read up on CTEs.

2. Setup a .psqlrc

You setup a bashrc, vimrc, etc. Why not do the same for Postgres. Some of the great things you can do:

Setup pretty formatting by default with \x auto
Set nulls to actually look like something \pset null ¤
Turn timing on by default \timing on
Customize your prompt \set PROMPT1 '%[%033[33;1m%]%x%[%033[0m%]%[%033[1m%]%/%[%033[0m%]%R%# '
Save commonly run queries that you can run by name

Here’s an example of my own psqlrc:

\set QUIET 1
\pset null '¤'
-- Customize prompts
\set PROMPT1 '%[%033[1m%][%/] # '
\set PROMPT2 '... # '
-- Show how long each query takes to execute
\timing
-- Use best available output format
\x auto
\set VERBOSITY verbose
\set HISTFILE ~/.psql_history- :DBNAME
\set HISTCONTROL ignoredups
\set COMP_KEYWORD_CASE upper
\unset QUIET

3. pg_stat_statements for where to index

pg_stat_statements is probably the single most valuable tool for improving performance on your database. Once enabled (with create extension pg_stat_statements) it automatically records all queries run against your database and records often and how long they took. This allows you to then go and find areas you can optimize to get overall time back with one simple query:

SELECT
(total_time / 1000 / 60) as total_minutes,
(total_time/calls) as average_time,
query
FROM pg_stat_statements
ORDER BY 1 DESC
LIMIT 100;

Yes, there is some performance cost to leaving this always on, but it’s pretty small. I’ve found it’s far more useful to be on and get major performance wins vs. the small cost of it always recording.

You can read much more on Postgres performance on a previous post

4. Slow down with ETL, use FDWs

If you have a lot of microservices or different apps then you likely have a lot of different databases backing them. The default for about anything you want to do is do create some data warehouse and ETL it all together. This often goes a bit too far to the extreme of aggregating everything together.

For the times you just need to pull something together once or on rare occasion foreign data wrappers will let you query from one Postgres database to another, or potentially from Postgres to anything else such as Mongo or Redis.

5. array and array_agg

There’s little chance if you’re building an app you’re not using arrays somewhere within it. There’s no reason you shouldn’t be doing the same within your database as well. Arrays can be just another datatype within Postgres and have some great use cases like tags for blog posts directly in a single column.

But, even if you’re not using arrays as a datatype there’s often a time when you want to rollup something like an array in a query then comma separate it. Something similar to the following could allow you to easily roll up a comma separated list of projects per user:

SELECT
users.email,
array_to_string(array_agg(projects.name), ',')) as projects
FROM
projects,
tasks,
users
WHERE projects.id = tasks.project_id
AND tasks.due_at > tasks.completed_at
AND tasks.due_at > now()
AND users.id = projects.user_id
GROUP BY
users.email

6. Use materialized views cautiously

If you’re not familiar with materialized view they’re a query that has been actually created as a table. So it’s a materialized or basically snapshotted version of some query or “view”. In their initial version materialized versions, which were long requested in Postgres, were entirely unusuable because when you it was a locking transaction which could hold up other reads and acticities avainst that view.

They’ve since gotten much better, but there’s no tooling for refreshing them out of the box. This means you have to setup some scheduler job or cron job to regularly refresh your materialized views. If you’re building some reporting or BI app you may undoubtedly need them, but their usability could still be advanced so that Postgres knew how to more automatically refresh them.

If you’re on Postgres 9.3, the above caveats about preventing reads still does exist

7. Window functions

Window functions are perhaps still one of the more complex things of SQL to understand. In short they let you order the results of a query, then compute something from one row to the next, something generally hard to do without procedural SQL. You can do some very basic things with them such as rank where each result appears ordered by some value, or something more complex like compute MoM growth directly in SQL.

8. A simpler method for pivot tables

Table_func is often referenced as the way to compute a pivot table in Postgres. Sadly though it’s pretty difficult to use, and the more basic method would be to just do it with raw SQL. This will get much better with Postgres 9.5, but until then something where you sum up each condition where it’s true or false and then totals is much simpler to reason about:

select date,
sum(case when type = 'OSX' then val end) as osx,
sum(case when type = 'Windows' then val end) as windows,
sum(case when type = 'Linux' then val end) as linux
from daily_visits_per_os
group by date
order by date
limit 4;

Example query courtesy of Dimitri Fontaine and his blog.

9. PostGIS

Sadly on this one I’m far from an expert. PostGIS is arguably the best option of any GIS database options. The fact that you get all of the standard Postgres benefits with it makes it even more powerful–a great example of this is GiST indexes which came to Postgres in recent years and offers great performance gains for PostGIS.

If you’re doing something with geospatial data and need something more than the easy to use earth_distance extension then crack open PostGIS.

10. JSONB

I almost debated leaving this one off the list, ever since Postgres 9.2 JSON has been at least one of the marquees in each Postgres release. JSON arrived with much hype, and JSONB fulfilled on the initial hype of Postgres starting to truly compete as a document database. JSONB only continues to become more powerful with better libraries for taking advantage of it, and it’s functions improving with each release.

If you’re doing anything with JSON or playing with another document database and ignoring JSONB you’re missing out, of course don’t forget the GIN and GiST indexes to really get the benefits of it.

The year ahead

Postgres 9.5/9.6 should continue to improve and bring many new features in the years ahead, what’s your preference for something that doesn’t exist yet but you do want to see land in Postgres. Let me know @craigkerstiens

searchcode server

Ben E. C. Boyter — Tue, 29 Dec 2015 04:17:42 GMT

A month or so ago I started collection emails on searchcode.com to determine if there was enough interest in a downloadable version of searchcode. The results were overwhelmingly positive. The email list grew far beyond what I would have expected, and this was in the first month. As such I have been working in this downloadable version of searchcode which will probably be called searchcode server.

Progress has been reasonably straight forward consider that searchcode.com is written using mostly Python and searchcode server is mostly Java. The main reason for choosing Java is that I really wanted searchcode server to be a self contained application which could be downloaded and run without the configuration and setup of additional services.

Kettle, my personal dashboard app

Ben Overmyer's Site — Tue, 29 Dec 2015 02:00:00 GMT

Not too long ago, I got fed up with having to use a ton of different apps to do various things online. Also, having to deal with apps that almost-but-not-quite fit the bill for what I was trying to do was a bit frustrating. So, I set out to write my own personal dashboard web app to handle… well, everything. I'm calling it Kettle, mostly because that was the first noun that popped into my head that sounded vaguely appropriate. Does it make sense? Not really. It just sounds right, much like J.R.R. Tolkien loved the sound of the phrase “cellar door.” At the time of this posting, I've just finished the first iteration of the note-taking part of the app. It doesn't yet allow searching or deleting notes, but other than that, it's perfectly usable. You can check it out at the Kettle repository on GitHub.

Windows vs. Linux Hardware Support

Justin Garrison's Homepage — Tue, 29 Dec 2015 02:00:00 GMT

Ludum Dare 34 Postmortem

Evan Todd — Mon, 28 Dec 2015 23:00:00 GMT

Friday 21:15

Fifteen minutes after the theme announcement, my friend Ben Homan walks through my front door. Not really my front door, I'm just a subletter. But this is a first. Normally he ignores our instructions to walk in without knocking. The first time, he texted me from the driveway.

21:30

Jesse Kooner walks in, also unannounced, bearing frozen pizza. Before he can even kick his shoes off, I loudly explain the theme: a never-before-seen tie between "growing" and "two-button controls".

Trying out Ceph with Oh-My-Vagrant

The Technical Blog of James on purpleidea.com — Mon, 28 Dec 2015 16:59:23 GMT

Daniel P. Berrangé wrote about trying out a single node ceph cluster. I decided to take his article and turn it into an Oh-My-Vagrant omv.yaml file. It took me about two minutes to do so, and two hours to debug a problem caused by something I had broken on my laptop.

If you’d like to replicate his article in less than 5 minutes, pull down the omv.yaml file that I’ve just published and run omv up. Here’s the full terminal output of my session:

App Use: Chordion

infrequent oscillations — Mon, 28 Dec 2015 16:34:20 GMT

Chordion is a combination arpeggiator / drum sequencer that supposedly “is a fun new way to make music on the iPad.” I think I’ve proven their “never hit a wrong note!” wrong. Maybe a few more practice runs will help. Listen on Soundcloud

Java volatile vs atomic variables

Studiofreya SSG Site — Mon, 28 Dec 2015 01:15:36 GMT

Volatile and atomic variables are an important feature in Java language. Especially when dealing with multi threading. A simple incrementation algorithm works fine in a single threaded program, but won’t work as expected in a multi threaded system:

private int counter = 0;

public int increment(int i) {
   return i++;
}

This code will work correctly in a single threaded program, but will give incorrect results in case of a multi threaded execution. How do we fix it? With the help of Java volatile and atomic variables.

Postgres 9.5 - The feature rundown

CRAIG KERSTIENS — Sun, 27 Dec 2015 22:55:56 GMT

The headline of Postgres 9.5 is undoubtedly: Insert… on conflict do nothing/update or more commonly known as Upsert or Merge. This removes one of the last remaining features which other databases had over Postgres. Sure we’ll take a look at it, but first let’s browse through some of the other features you can look forward to when Postgres 9.5 lands:

Grouping sets, cube, rollup

Pivoting in Postgres has sort of been possible as has rolling up data, but it required you to know what those values and what you were projecting to, to be known. With the new functionality to allow you to group various sets together rollups as you’d normally expect to do in something like Excel become trivial.

So now instead you simply add the grouping type just as you would on a normal group by:

SELECT department, role, gender, count(*)
FROM employees
GROUP BY your_grouping_type_here;

By simply selecting the type of rollup you want to do Postgres will do the hard work for you. Let’s take a look at the given example of department, role, gender:

grouping sets will project out the count for each specific key. As a result you’d get each department key, with other keys as null, and the count for each that met that department.
cube will give you the same values as above, but also the rollups of every individual combination. So in addition to the total for each department, you’d get breakups by the department and gender, and department and role, and department and role and gender.
rollup will give you a slightly similar version to cube but only give you the detailed groupings in the order they’re presented. So if you specified roll (department, role, gender) you’d have no rollup for department and gender alone.

Check the what’s new wiki for a bit more clarity on examples and output

Import foreign schemas

I only use foreign tables about once a month, but when I do use them they’ve inevitably saved many hours of creating a one off ETL process. Even still the effort to setup new foreign tables has shown a bit of their infancy in Postgres. Now once you’ve setup your foreign database, you can import the schema, either all of it or specific tables you prefer.

It’s as simple as:

IMPORT FOREIGN SCHEMA public
FROM SERVER some_other_db INTO reference_to_other_db;

pg_rewind

If you’re managing your own Postgres instance for some reason and running HA, pg_rewind could become especially handy. Typically to spin up replication you have to first download the physical, also known as base, backup. Then you have to replay the Write-Ahead-Log or WAL–so it’s up to date then you actually flip on replication.

Typically with databases when you fail over you shoot the other node in the head or STONITH. This means just get rid of it, completely throw it out. This is still a good practice, so bring it offline, make it inactive, but from there now you could then flip it into a mode and use pg_rewind. This could save you pulling down lots and lots of data to get a replica back up once you have failed over.

Upsert

Upsert of course will be the highlight of Postgres 9.5. I already talked about it some when it initially landed. The short of it is, if you’re inserting a record and there’s a conflict, you can choose to:

Do nothing
Do some form of update

Essentially this will let you have the typically experience of create or update that most frameworks provide but without a potential race condition of incorrect data.

JSONB pretty

There’s a few updates to JSONB. The one I’m most excited about is making JSONB output in psql read much more legibly.

If you’ve got a JSONB field just give it a try with:

SELECT jsonb_pretty(jsonb_column)
FROM foo;

Give it a try

Just in time for the new year the RC is ready and you can get hands on with it. Give it a try, and if there’s more you’d like to hear about Postgres please feel free to drop me a note craig.kerstiens@gmail.com.

Joel Califa: Product Design Lead at Digital Ocean

Sam Solomon — Sun, 27 Dec 2015 02:00:00 GMT

How can companies build great design teams and what do those teams look like?

Going from blog posts to full launches

CRAIG KERSTIENS — Sat, 26 Dec 2015 22:55:56 GMT

I recall extremely early stage where you’d build a feature, realize it was awesome, then the next day write a blog post for it. At some point you start to move from that to more coordinated launches. A larger coordinated launch allows you to reach a bigger audience, can lead to bigger deals, and help expand your overall market. But perhaps more importantly by the time you hit full launch you’ve message tested and ensured it’s going to resonate in the way you expect.

The process itself will both help amplify and validate/refine your message

This is often a more gradual process than a sudden single change, you’ll introduce new parts of this in time. And for many what an entire launch process looks like comes by trial an error, to help shorten that learning curve here’s key areas I pay attention for a launch and process followed by a rough timeline.

Product first

Making sure the product is in the right shape is key to any big launch. You don’t get a second shot and if the product isn’t in shape customers often won’t take a second look at it later. For this reason I strongly prefer to have your product locked and loaded before you even start talking launch times, or at least be in the bug clean up phase. This means you’ve built a feature, validated with alpha users or private beta, and are ready to open it up to the world.

If you have to set a launch date without the product or feature being already done allow padding. Sometimes it’s good for the team to know the padding, sometimes it isn’t. When you have extra time it’s not uncommon for your development to magically consume exactly that amount of time and still result in a small scramble towards the end.

A good driver I’ve found is needing to have it fully like to demo a few weeks out from the launch itself, such as during analyst pre-briefings.

Crafting your message

Every launch is an opportunity to tell your core message and value prop. If you miss this opportunity for focusing on a single narrow feature you’ve missed the biggest opportunity you had in a launch. You can’t relaunch your full product every time though–you do need some big improvements or feature that you can highlight, but you should still hit your core message.

First you should know that your feature solves some specific problem, you should know this from the alpha/beta testing and if it doesn’t solve this problem you’re not ready to launch. Yes, some people will launch a product before the product is completely there–this is common in a marketing driven company as opposed to a product/engineering driven company.

Your message should lead with the problem you’re solving, not the laundry list of features. The best launches lead with some broader thematic message, even better if it’s an altruistic world changing one. A rough example of this:

To the point to the product, and probably over generalized as boring:

Connectify brings a new way of taking your dumb devices at home and turning them into intelligent connected devices.

In contrast, broad thematic message, followed lightly by the product.

We live in a connected world, and with new connected devices there’s the opportunity not just give you more data but help you improve how you live your life. Connectify helps you at bringing the devices that matter together with ease.

Testing your message

You should treat your message just like a product, testing it gradually along the way. Once you’ve got some initial framing of it, test it internally, then with friendly customers or community members. Leading up to the launch I usually have a timeline and get all the content and communication rolling about 3 weeks out. I’ll give a bit of a timeline below but first some more around message testing

Analysts

If you regularly use any analysts you should absolutely use them to help with a launch. Several weeks out is a great time to test key messages with them, get feedback, and if you’re lucky you may even get them to provide a quote for the launch.

Keep in mind here a inquiry is an opportunity to test your message and get feedback. You should talk roughly half of the time here, they should be talking the other half. In contrast briefings before launch you should have your message fully baked and should simply be pitching your message and possibly demo-ing.

Friendlies

This may be more contentious, but at least at early stage sharing drafts with friendly community members is a great way to get feedback and refine your message. Here you should be especially concious of the request of their time and expect to have some delay before they get back to you. Being top secret about your message ahead of time won’t add much value to it being a home run, where as better ensuring it resonates will help it to be more successful.

Customers

Customers I call out as a separate bucket. Customers have less incentive to leak your news than friendlies, but also fall somewhere on the other spectrum of analysts. A key piece about customers is there is an opportunity for them to be a launch partner. And so on to that topic:

Launch partners

Press and others like seeing and knowing you have external validation. Similarly many see the benefit of being part of a launch, after all it’s more free press for them. For a launch partner there are various levels, though for most providing some quote is a pretty common level.

The best way to do this is talk to them about what they like about the feature/product and take a first stab at the quote for them from their feedback. Some may very much want to wordsmith their own which is fine, but minimizing the work required of them while–trying to hit something they’d say as well–as a message that flows can best be done by you taking a first pass.

Further there’s varying levels of value with quotes and references. In descending order:

Customers
Analysts
Community Members

The other details

During launch week I mostly want to be dotting i’s and crossing t’s, meaning: I want the product done. I want documentation done. I want the blog post finalized. I want to be in the mode of send internal announcements, prep internal teams, talk to media.

Prepping internal teams

Obviously the engineering and product people involved will be in the loop. But you need to notify many others some of which should have been in the loop already, some less so:

Support - There’s a new product surface area, support should be top of your list so they can field the tickets and questions that come in
Sales - Even if there is no price change or impact, new features allow sales to communicate value to customers

Timeline

Finally what’s the end to end timeline look like with all the little details. Here’s a rough one that’s fully built out. IF you’re smaller and don’t have a regular cadence of analysts in hand then just expect that doesn’t apply. IF your support team is the product and engineers maybe that’s lighter weight. Basically feel free to take out parts, but expect your process to grow to something of this size.

4 weeks out - Outline of blog post with key messages
Test that outline internally
3 weeks out - Start to get a rough draft in place
3 weeks out - Share internally and with friendlies. At this point you’re explicitely looking for message feedback. Tell people to not waste time on nitpicks of words or grammar, it will be 98% re-written by the time you’re done
2.5 weeks out - Analyst inquiries for message testing
2.5 weeks out - Start putting together product demo
2 weeks out - Start putting together documentation
2 weeks out - Start nailing down blog post for final messages
1 weeks out - Start to put final touches on blog post for grammar
1 week out - Analyst briefings
1 week out - Update support
3-5 days out - Stage blog post
3-5 days out - Stage new documentation
2-4 days out - Make sure PRs are ready or feature flags, in short the switch is there or live but not public
1-3 days out - Update sales
1-3 days out - Interall communication to all@
1-3 days out - Media briefings
LAUNCH DAY - Sit in a room and watch all the things, engage with twitter/HN/etc.

App Use: Keezy Drummer

infrequent oscillations — Sat, 26 Dec 2015 17:47:08 GMT

A quick noodle with Keezy Drummer recorded into the Zoom H4n, exported to Fission, then piped through Audio Hijack with the right channel duplicated into the left before saving to MP3. Listen on Soundcloud

Janathon, Dryathlon, Jamathon

infrequent oscillations — Sat, 26 Dec 2015 16:04:49 GMT

And a partridge in a pear tree. Signed up for Janathon (updates will probably be here). Signed up for Dryathlon (Sponsor me!). Decided to do an ad-hoc Jamathon (“Make and upload some musical output every day using one or two mobile apps”). (Yes, I’m going to be insufferable all month.)

Monty Python and the Holy Grail

🦉 olshansky 🦁 — Sat, 26 Dec 2015 13:55:24 GMT

Olshansky's review of Monty Python and the Holy Grail

App Use: Audioshare

infrequent oscillations — Sat, 26 Dec 2015 13:36:41 GMT

Not strictly a music-making app, [Audioshare] gets used for almost all of the things I upload to Soundcloud. But since I managed to accidentally make some “music” with it, I might as well blog about it (and steal the description from Soundcloud). There I was, playing something I’d just made in Soundscaper through Audioshare via Audiobus into Gliderverb when I had the urge to record what was happening since it was sounding ok.

App Use: TC-Performer

infrequent oscillations — Sat, 26 Dec 2015 13:33:27 GMT

Like a 2D theremin on acid, TC-Performer has a bunch of instruments (all digitally synthed by TC-11 - which is on the wishlist) controlled by multi-touch, movement, etc. Bit tricky to do coherent tunes but excellent for making this kind of swoopy droney chimey kind of space music. Listen on Soundcloud

App Use: Poly

infrequent oscillations — Sat, 26 Dec 2015 13:29:35 GMT

Some learning-curve twiddling with Poly. Should be good for generating driving rhythms as underlying texture. Listen on Soundcloud

App Use: Fieldscaper

infrequent oscillations — Sat, 26 Dec 2015 13:25:31 GMT

Recorded 1:45 of tap filling a bath using Fieldscaper. Chopped out a decent 45s segment and mangled it three different ways to make a kind of weird Dr Who 1970s sci-fi noisefest. Listen on Soundcloud

Insomniac Art

infrequent oscillations — Sat, 26 Dec 2015 13:06:09 GMT

Or “half an hour spent twiddling with various graphics apps on an iPhone whilst unable to sleep”. (Lots of 512x512 images make this a big post.) The original image Southwark Park, 2015-12-19, near the south-east gate, looking south-west. Waterlogue “Vibrant, medium, Auto”. BLACK “P400, Tura expired 2002” BLACK plus Waterlogue Combination of the last two. Decim8 These were all done via the random settings button. Etchings plus Union plus Pixlr Etchings was used to create a few variants of the BLACK version which where then combined in Union before being tweaked in Pixlr.

Pi-Hole for Ubuntu 14.04

Ben E. C. Boyter — Sat, 26 Dec 2015 04:24:32 GMT

Because of the fact that I personally work for an ad supported company and that searchcode.com is currently supported via third party advertising I tend to keep an eye on the state of ad blockers on the web.

Most people probably know about adblockplus and other browser extensions however there are other ways to block ad’s on ones network. One that I had previously read about was setting up your own Bind9 server on a server and adding custom rules to block them at a DNS level. Other the last week I had been playing around with this but since I am not a bind expert I was unable to get it working in a satisfactory way.

Indian startups ecosystem: Fear of failure grounds our success story

Blog on Kailash Nadh / Personal homepage — Sat, 26 Dec 2015 02:00:00 GMT

The Indian startup ecosystem has taken shape and exploded in the last few years, and so have the countless stories surrounding them. Unsurprisingly, lessons of unsuccessful ideas and attempts haven’t gotten as much precedence as successful counterparts.

Clojure bookmarks

xenodium.com @alvaro — Fri, 25 Dec 2015 02:00:00 GMT

William Channer: Host of DRT, Founder of Panda

Sam Solomon — Fri, 25 Dec 2015 02:00:00 GMT

William Channer is a partner at the Panda Network, and the host of Dorm Room Tycoon.

Shine a Spotlight On Those Photos

Beetle Space — Thu, 24 Dec 2015 10:00:00 GMT

A nearby photography club has annual county wide competitions. The judges select their favorite photos and display them in a mall for a month.

Several years ago, I took a friend to the mall to show him the photos. We had only a few minutes until closing time, and they …

Baby steps with `libclang`: Walking an abstract syntax tree

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Wed, 23 Dec 2015 19:04:29 GMT

Recently, I wanted to parse some C++ code to collect some information about my coding style. As always with C++, this turned out to require more effort than I had originally anticipated. Apparently, C++ is so complex that you require more or less a compiler to parse it. The great folks at the LLVM project had already done the work—a stroke of luck for me.

So I set out to use clang for parsing C++ code. The official documentation lists multiple options here. While LibTooling, a C++ interface to clang, would arguably have been the more appropriate choice, I settled for libclang, a C interface. Its main advantage is its stability. The LLVM developers are very reluctant to introduce breakages here (although they have been known to happen). The disadvantage of libclang is that we do not get full control over the abstract syntax tree (AST) of the program. I can live with that, I guess.

In this post, I only want to briefly introduce some basic concepts of libclang. I hope to show off more features in subsequent posts.

Dumping an AST

Suppose you have the following simple C++ snippet:

template <class T> bool f( T x )
{
  return x % 2;
}

Just like any compiler, clang creates an abstract syntax tree that represents the code we want to compile. The syntax of this AST resembles the written code very closely, which makes this a very interesting view on your code.

To show the AST of the previous snippet, we can clang with:

clang -Xclang -ast-dump -fsyntax-only foo.cc

This results in the following output:

TranslationUnitDecl 0x2fef1b0 <<invalid sloc>> <invalid sloc>
|-TypedefDecl 0x2fef6f0 <<invalid sloc>> <invalid sloc> implicit __int128_t '__int128'
|-TypedefDecl 0x2fef750 <<invalid sloc>> <invalid sloc> implicit __uint128_t 'unsigned __int128'
|-TypedefDecl 0x2fefb10 <<invalid sloc>> <invalid sloc> implicit __builtin_va_list '__va_list_tag [1]'
`-FunctionTemplateDecl 0x2fefdc0 <foo.cc:1:1, line:4:1> line:1:25 f
  |-TemplateTypeParmDecl 0x2fefb60 <col:11, col:17> col:17 referenced class T
  `-FunctionDecl 0x2fefd20 <col:20, line:4:1> line:1:25 f '_Bool (T)'
    |-ParmVarDecl 0x2fefc20 <col:28, col:30> col:30 referenced x 'T'
    `-CompoundStmt 0x2fefea0 <line:2:1, line:4:1>
      `-ReturnStmt 0x2fefe80 <line:3:3, col:14>
        `-BinaryOperator 0x2fefe58 <col:10, col:14> '<dependent type>' '%'
          |-DeclRefExpr 0x2fefe10 <col:10> 'T' lvalue ParmVar 0x2fefc20 'x' 'T'
          `-IntegerLiteral 0x2fefe38 <col:14> 'int' 2

The important thing to note is that we have a FunctionTemplateDecl node that contains, among others, the body of the function (starting from the CompoundStmt node).

To familiarize ourselves with libclang, let’s try to reproduce parts of this AST ourselves. I propose writing a program that walks the AST and prints out node names.

First, we need to get clang to emit the AST in a binary format so that we traverse it later:

clang++ -emit-ast foo.cc

This should result in a file foo.ast.

Reading an AST with `libclang`

Understanding libclang requires reading copious amounts of documentation. I cannot claim in good conscience that I have fully understood and memorized every detail, so the code that follows may actually be very stupid.

One of the core functions of libclang is the creation of an index. This data structure contains multiple translation units, e.g. source files, that make up an executable. Given an index, a new translation unit can be created. There are various ways of doing this. It it possible to parse source code directly. For now, I will only present a way of parsing a dumped AST. Last, we need a cursor, which is the libclang term for a thing that keeps track of where we are in the AST.

The following snippet will create the required things for us and permit further parsing (for the hasty: scroll to the bottom for the complete example code):

#include <clang-c/Index.h>

int main( int argc, char** argv )
{
  if( argc < 2 )
    return -1;

  CXIndex index        = clang_createIndex( 0, 1 );
  CXTranslationUnit tu = clang_createTranslationUnit( index, argv[1] );

  if( !tu )
    return -1;
  
  CXCursor rootCursor  = clang_getTranslationUnitCursor( tu );

  clang_disposeTranslationUnit( tu );
  clang_disposeIndex( index );
  return 0;
}

This program will attempt to load a translation unit from an .ast file, get a cursor to the root node, and exit. The only noteworthy thing are the two _dispose calls at the end of the function. This is due to libclang being a C library—we actually need to think about memory management here. Darn.

Visiting nodes

To make the example application marginally more useful, we need to make use of the “visitor pattern”. libclang requires us to specify a sort of callback function that is called whenever we reach a node in the AST.

The function needs to have the following signature:

CXChildVisitResult visitor( CXCursor cursor, CXCursor parent, CXClientData clientData )

The result of the function decides how the traversal is being continued. Possibilities include CXChildVisit_Break to stop traversal, CXChildVisit_Continue to continue traversing the siblings of the current cursor without visiting its children, or CXChildVisit_Recurse to visit the children of the current cursor first. cursor refers to the current position in the tree. parent is the parent node in the tree. Last, CXClientData is a typedef for ‘void* (a, don’t you love C as much as I do?) and permits passing additional information for each traversal. We will soon figure out a way of using this to our advantage.

The main entry point for visiting nodes is the clang_visitChildren function. Assuming that we have set up a function visitor with the proper prototype, we may add the following lines after rootCursor has been assigned:

unsigned int treeLevel = 0;

clang_visitChildren( rootCursor,
                     visitor,
                     &treeLevel );

This will call our visitor function with client data that initially refers to the upper-most level in the tree. A potential implementation for visitor could look like this:

CXChildVisitResult visitor( CXCursor cursor, CXCursor /* parent */, CXClientData clientData )
{
  CXSourceLocation location = clang_getCursorLocation( cursor );
  if( clang_Location_isFromMainFile( location ) == 0 )
    return CXChildVisit_Continue;

  CXCursorKind cursorKind = clang_getCursorKind( cursor );

  unsigned int curLevel  = *( reinterpret_cast<unsigned int*>( clientData ) );
  unsigned int nextLevel = curLevel + 1;

  std::cout << std::string( curLevel, '-' ) << " " << getCursorKindName(
  cursorKind ) << " (" << getCursorSpelling( cursor ) << ")\n";

  clang_visitChildren( cursor,
                       visitor,
                       &nextLevel ); 

  return CXChildVisit_Continue;
}

The visitor uses some interesting functions. First, we use clang_getCursorLocation() to get the location of the cursor with respect to the source file. This permits mapping a cursor, i.e. a region in a tree, to a region in the source file. We use it for a far simpler purpose, though. By calling clang_Location_isFromMainFile(), we can check whether a node is part of the source file we are analysing or not. This is extremely helpful for skipping over nodes that are not interesting for us because they appear in the AST after an #include, for example. As you can see, we skip over the subtree rooted at the current cursor when such a node has been reached.

The next interesting thing is the clang_getCursorKind. This yields the type of the cursor, giving us information about whether it refers to a statement, a function declaration, or what have you. To see all types (or rather kinds) that are available, please take a look at the Index.h file.

Having obtained information about the type of the cursor, we get information about our current level, using a reinterpret_cast. We then call two auxiliary functions that give us the name and spelling of the cursor (see below for whole code). By spelling, libclang means the name of the entity the cursor refers to. When positioned at the root node of a function declaration, the cursor spelling will be the name of the function, for example. Not all entities have a spelling—for some, we need to tokenize the node first. For reasons of simplicity, I am going to cover this in a later post, though.

The last interesting thing is the continued traversal within the visitor function. We call clang_visitChildren on the current cursor, with the updated level information. Thus, all children of the current cursor will eventually be visited. Last, we return CXChildVisit_Continue. This ensures that we do not visit the children of a particular node twice.

The output of this simple program when being applied to foo.ast is:

 FunctionTemplate (f)
- TemplateTypeParameter (T)
- ParmDecl (x)
-- TypeRef (T)
- CompoundStmt ()
-- ReturnStmt ()
--- BinaryOperator ()
---- DeclRefExpr (x)
---- IntegerLiteral ()

Which is nice for just about 50 lines of code in total. As promised, not all entities have names here, but the ones that do seem to make sense.

The complete code

Here’s the complete code of this example:

#include <clang-c/Index.h>

#include <iostream>
#include <string>

std::string getCursorKindName( CXCursorKind cursorKind )
{
  CXString kindName  = clang_getCursorKindSpelling( cursorKind );
  std::string result = clang_getCString( kindName );

  clang_disposeString( kindName );
  return result;
}

std::string getCursorSpelling( CXCursor cursor )
{
  CXString cursorSpelling = clang_getCursorSpelling( cursor );
  std::string result      = clang_getCString( cursorSpelling );

  clang_disposeString( cursorSpelling );
  return result;
}

CXChildVisitResult visitor( CXCursor cursor, CXCursor /* parent */, CXClientData clientData )
{
  CXSourceLocation location = clang_getCursorLocation( cursor );
  if( clang_Location_isFromMainFile( location ) == 0 )
    return CXChildVisit_Continue;

  CXCursorKind cursorKind = clang_getCursorKind( cursor );

  unsigned int curLevel  = *( reinterpret_cast<unsigned int*>( clientData ) );
  unsigned int nextLevel = curLevel + 1;

  std::cout << std::string( curLevel, '-' ) << " " << getCursorKindName(
  cursorKind ) << " (" << getCursorSpelling( cursor ) << ")\n";

  clang_visitChildren( cursor,
                       visitor,
                       &nextLevel ); 

  return CXChildVisit_Continue;
}

int main( int argc, char** argv )
{
  if( argc < 2 )
    return -1;

  CXIndex index        = clang_createIndex( 0, 1 );
  CXTranslationUnit tu = clang_createTranslationUnit( index, argv[1] );

  if( !tu )
    return -1;

  CXCursor rootCursor  = clang_getTranslationUnitCursor( tu );

  unsigned int treeLevel = 0;

  clang_visitChildren( rootCursor, visitor, &treeLevel );

  clang_disposeTranslationUnit( tu );
  clang_disposeIndex( index );

  return 0;
}

I am releasing the code into the public domain. Don’t forget to link against libclang when compiling it (subsequent posts will provide a find module for CMake). Should you consider this code useful, it would give me enormous pleasure if you were to drop me an e-mail.

To show that this would be worth your while, I am pointing out an interesting part in the code above: In the getCursorKindName() function, we use clang_getCursorKindSpelling() to obtain the name of the type the current cursor refers to. This is a CXString, which needs to be disposed later on with clang_disposeString() so as not to cause any memory leaks. To use the string as a regular C++ string, we need to call the helper function clang_getCString, which returns a null-terminated C string (i.e. a const char*, which we may finally assign to an std::string).

Wishing you a happy tree traversal! Maybe clang hides some gifts in the AST…

Surprises with Java packages

GeekWare - Daniel Pecos Martínez — Wed, 23 Dec 2015 17:30:00 GMT

Packages in Java is a quite simple and straightforward concept of the language. It’s there from the beginning and it’s commonly used by every Java programmer. In a few words, these are the rules you have to follow to create a class inside a package (spoiler: which are not completely true, as we’ll see later):

Package statement must be the first one specified in a java class file
A package namespace must match the physical path of the file, i.e a class defined with a package a.b.c must be placed in a path a/b/c/

Easy right? Let’s check…

Turns out that the second rule is not mandatory but a recommendation (a good one, BTW), but Eclipse IDE enforces it to the point that if you don’t follow it, it throws a compilation error. Let’s open the following class in Eclipse:

package a.b.c;

public class HelloWorld {
 public static void main(String[] args) {
   System.out.println("Hello world!");
 }
}

And you’ll get the following error:

But as I said, it’s not a restriction of the language but from the IDE itself:

In fact, turns out that manually compiling this class works perfectly fine (also tried maven and worked flawesly):

$ javac src/HelloWorld.java -d bin

$ java -cp bin a.b.c.HelloWorld
Hello world!

$ tree bin/
bin/
└── a
    └── b
        └── c
            └── HelloWorld.class

Corollary: if you’re a programmer, don’t trust even the most basic and accepted statement you could think of. Even if you’re absolutely right at this very moment, it will eventually be modified and surprise you.

App Use: Soundscaper

infrequent oscillations — Wed, 23 Dec 2015 11:30:52 GMT

The samples from Disquiet Junto 0207 soundscaped using the excellent SoundScaper (mostly) controlled by a KORG nanoKONTROL2. Listen on Soundcloud

App Use: Phase Rings, Dedalus

infrequent oscillations — Wed, 23 Dec 2015 11:25:31 GMT

A simple bit of noodling in Phase Rings (I think it was a Mixolydian scale but I can’t remember now - Always Takes Notes At The Time, Kids) through Dedalus for some texture. Listen on Soundcloud

App Use: Patterning, Dedalus

infrequent oscillations — Wed, 23 Dec 2015 11:18:45 GMT

A simple drum loop in Patterning filtered through Dedalus whilst I scrub through the various options to see what kind of aural mess I can make (hint: “A big one”). Good learning experience though. Listen on Soundcloud

Links - December 22nd, 2015

Avy Faingezicht — Tue, 22 Dec 2015 02:00:00 GMT

Luigi review

Bytepawn - Marton Trencseni — Tue, 22 Dec 2015 01:00:00 GMT

I review Luigi, an execution framework for writing data pipes in Python code. It supports task-task dependencies, it has a simple central scheduler with an HTTP API and an extensive library of helpers for building data pipes for Hadoop, AWS, Mysql etc. It was written by Spotify for internal use and open sourced in 2012. A number of companies use it, such as Foursquare, Stripe, Asana.

Cargo Cult Data

Bytepawn - Marton Trencseni — Tue, 22 Dec 2015 01:00:00 GMT

Cargo cult data is when you're collecting and looking at data when making decisions, but you're only following the forms and outside appearances of scientific investigation and missing the essentials, so it doesn't work.

The brain scope is growing: BrainHub, BrainRunners, Brains

Liza Shulyayeva — Mon, 21 Dec 2015 20:17:06 GMT

I’ve been MIA for a while, I know. I haven’t stopped working on SnailLife - I just wanted to get to some sort of completion with the brain refactoring phase before talking about it. Now, however, I realize there won’t be any completion for a long while and I may as well post an update.

List of Awesomeness #4

THINK@RJ — Mon, 21 Dec 2015 17:07:00 GMT

In this installation I won’t be providing any book or paper suggestions. Instead I will summarize some of my thoughts on the previous list.

Papers

Not adding a new serious paper this week in order to give time to those who are still on Mr. Einstein’s

Disquiet Junto #207

infrequent oscillations — Mon, 21 Dec 2015 14:30:15 GMT

Screenshots of my effort for the Disquiet Junto #207 challenge. “M1 arpeggiated” - the underlying heavy drums “M2 bowed” - the gentle machine-like throbbing “M3 bowed” - the occasional background swell “M1 taped” - the initial drum-style intro “M2 arpeggiated” - extra texture “M3 shuffled” - the rolling trill (cf 2:40, 3:35)

Mac OS X tips backlog

xenodium.com @alvaro — Mon, 21 Dec 2015 02:00:00 GMT

[TODO]{.todo .TODO} Uebersicht: Keep an eye on what is happening on your machine and in the World.

[DONE]{.done .DONE} Kwm: Tiling window manager with focus follows mouse for OSX.

cp ~/homebrew/Cellar/kwm/1.1.3/homebrew.mxcl.kwm.plist ~/Library/LaunchAgents/
launchctl load ~/Library/LaunchAgents/homebrew.mxcl.kwm.plist

[DONE]{.done .DONE} Turn off shadows with ShadowToggle.

[DONE]{.done .DONE} Disk Inventory X: disk usage utility for Mac.

Links - December 21st, 2015

Avy Faingezicht — Mon, 21 Dec 2015 02:00:00 GMT

Search bash history with Emacs helm

xenodium.com @alvaro — Sun, 20 Dec 2015 02:00:00 GMT

Following up from changing CWD with helm projectile, here's a way to search your bash history with helm:

(defun ar/helm-helm (title candidates on-select-function)
  "Helm with TITLE CANDIDATES and ON-SELECT-FUNCTION."
  (helm :sources `((name . ,title)
                   (candidates . ,candidates)
                   (action . ,on-select-function))
        :buffer "*helm-exec*"
        :candidate-number-limit 10000))

(defun ar/shell-send-command (command)
  "Send COMMAND to shell mode."
  (assert (string-equal mode-name "Shell") nil "Not in Shell mode")
  (goto-char (point-max))
  (comint-kill-input)
  (insert command)
  (comint-send-input))

(defun ar/helm-shell-search-history ()
  "Narrow down bash history with helm."
  (interactive)
  (assert (string-equal mode-name "Shell") nil "Not in Shell mode")
  (ar/helm-helm "bash history"
                (with-temp-buffer
                  (insert-file-contents "~/.bash_history")
                  (reverse
                   (delete-dups
                    (split-string (buffer-string) "\n"))))
                #'ar/shell-send-command))

Bonus: Replace existing M-r binding to use ar/helm-shell-search-history.

(bind-key "M-r" #'ar/helm-shell-search-history shell-mode-map)

Medicine bookmarks

xenodium.com @alvaro — Sat, 19 Dec 2015 02:00:00 GMT

Science-Based Medicine – Exploring issues and controversies in science and technology.

View DICOM files from your X-ray

xenodium.com @alvaro — Sat, 19 Dec 2015 02:00:00 GMT

Got a CD with my chest X-ray from the hospital. Was expecting a pdf or an image of sorts, but the CD content was rather different. For starters, it was targeted at Windows users (AUTORUN.INF, MediaViewerLauncher.EXE and a bunch of DLLs):

$ find . -exec file --mime-type '{}' \;

./AUTORUN.INF: text/plain
./DICOMDIR: application/dicom
./MediaViewerLauncher.EXE: application/octet-stream
...
./Libraries/BASEPRINTER.DLL: application/octet-stream
./Libraries/CDDATABURNER.DLL: application/octet-stream
./Libraries/COM.DLL: application/octet-stream
...
./Libraries/ACE.DLL: application/octet-stream
./Libraries/ACE_SSL.DLL: application/octet-stream
./Libraries/ATL90.DLL: application/octet-stream
...
./DICOM/PAT_0000: application/x-directory
./DICOM/PAT_0000/STD_0000/SER_0000/OBJ_0001/IM_0001: application/dicom
./DICOM/PAT_0000/STD_0000/SER_0001/OBJ_0001/ED_0001: application/dicom
./DICOM/PAT_0000/STD_0000/SER_0002/OBJ_0001/ED_0001: application/dicom
./Worklist/ClinicalInfo/067eccde-b299-e511-9114-005056ad3afe.mht: text/html
./Worklist/Report/067eccde-b299-e511-9114-005056ad3afe.mht: text/html
./Worklist/Worklist.wl: application/octet-stream

I'm on a Mac, so most of these files were not useful to me. The more interesting files were IM_0001 and ED_0001 with "application/dicom" MIME type. DICOM files stand for Digital Imaging and Communications in Medicine. How to view these on a Mac? OsiriX viewer is an option. OsiriX, though on the heavy side (100.7MB download), it rendered the X-ray successfully.

Unsurprisingly, ImageMagick's convert utility also handles DICOM files. Converting to PNG worked well.

$ convert ./DICOM/PAT_0000/STD_0000/SER_0001/OBJ_0001/ED_0001 ED_0001.png

DICOM files also hold patient's metadata and optional reports. The file format is well known. OsiriX gives you access to it, but a few lines of python can also extract it for you. First install the pydicom package:

$ sudo pip install pydicom

Running the python interpreter is enough to peek at the metadata:

>>> import dicom
>>> ds = dicom.read_file("./DICOM/PAT_0000/STD_0000/SER_0000/OBJ_0001/IM_0001")
>>> ds

(0008, 0000) Group Length                        UL: 400
(0008, 0005) Specific Character Set              CS: 'ISO_IR 100'
(0008, 0016) SOP Class UID                       UI: Computed Radiography Image Storage
(0008, 0020) Study Date                          DA: '20151203'
(0008, 0021) Series Date                         DA: '20151203'
(0008, 0023) Content Date                        DA: '20151203'
(0008, 0030) Study Time                          TM: '120519.000000'
(0008, 0031) Series Time                         TM: '120520.000000'
(0008, 0033) Content Time                        TM: '120643.000000'
(0008, 0060) Modality                            CS: 'CR'
(0008, 0070) Manufacturer                        LO: 'Canon Inc.'
...

There were other DICOM files with a report:

>>> import dicom
>>> ds = dicom.read_file("./DICOM/PAT_0000/STD_0000/SER_0001/OBJ_0001/ED_0001")
>>> ds

(0008, 0005) Specific Character Set              CS: 'ISO_IR 100'
(0008, 0016) SOP Class UID                       UI: Encapsulated PDF Storage
...
(0042, 0012) MIME Type of Encapsulated Document  LO: 'application/pdf'

DCMTK is another alternative tool to extract DICOM metadata. The source is available and can be built:

$ tar xf dcmtk-3.6.0.tar.gz
$ cd dcmtk-3.6.0
$ cmake .
$ make

Or installed via homebrew:

$ brew install dcmtk

DCMTK includes dcmdump. You can use it to dump DICOM files:

$ dcmdata/apps/dcmdump DICOM/PAT_0000/STD_0000/SER_0000/OBJ_0001/IM_0001

# Dicom-File-Format

# Dicom-Meta-Information-Header
# Used TransferSyntax: Little Endian Explicit
(0002,0000) UL 192                                      #   4, 1 FileMetaInformationGroupLength
(0002,0001) OB 01\00                                    #   2, 1 FileMetaInformationVersion
(0002,0002) UI =ComputedRadiographyImageStorage         #  26, 1 MediaStorageSOPClassUID
(0002,0003) UI [1.2.392.200046.100.2.1.1.42667.20151203120519.1.1.1] #  52, 1 MediaStorageSOPInstanceUID
(0002,0010) UI =LittleEndianExplicit                    #  20, 1 TransferSyntaxUID
(0002,0012) UI [1.3.46.670589.42.1.4.4.5]               #  24, 1 ImplementationClassUID
(0002,0013) SH [PhilipsISPACS445]                       #  16, 1 ImplementationVersionName
...

Of interest, David Clunie's Medical Image Format Site.

App Use: Samplr #2, Addictive Synth

infrequent oscillations — Fri, 18 Dec 2015 14:05:27 GMT

App Use, Track 3 is more experimentation with Samplr. “Sherlock Holmes and the Voice Of Terror” provided the samples, Fission chopped them up, and Addictive Synth provided the arpeggiation samples. This was pretty tricky. (There was an earlier attempt at this but it was a fair old disaster. Even more repetitive and far too “chunky”.) It took many attempts to get the first sample working right - Samplr loops the sample when recording in Tape Mode and cutting it off whilst keeping within the beat of a bar was a nightmare.

App Use: Portable Dandy, Loopy

infrequent oscillations — Fri, 18 Dec 2015 13:48:46 GMT

Portable Dandy is a collection of samples you play in a virtual tape loop with a trio of effects. Quite fun but it has a proper problem as you can hear in thesoundtestroom review - certain sample and effect combinations will trigger horrible glitching. In an attempt to avoid this horror, I recorded the samples into [LPR][Loopy] and then sequenced them there. Where I discovered that Loopy is a right old bugger to work for this kind of thing.

App Use: Samplr, #1

infrequent oscillations — Fri, 18 Dec 2015 13:23:42 GMT

Samplr is a wizard’s toolbox of magical sound manipulation. In the hands of a master, it can produce some good stuff - “Do It Again”, “jamming with Samplr”, “SAMPLR - have some fun”. In the hands of a bumbling amateur, it makes … something? App Use, Track 1 is just a bunch of stock Samplr samples being randomly noodled through the various modes to see how it all works. It works as a mostly ambient track, I think.

Pay Down Mortgage or Invest?

Beetle Space — Fri, 18 Dec 2015 10:00:00 GMT

The following scenarios are ones many of us run in to:

Lumped Sum: I suddenly got $30,000. Should I put the money towards my mortgage, or should I invest it?
Periodic Payments: After all my monthly bills, I have $500 left over. Should I put it all towards my …

Tip: GOOGLETRANSLATE your Spreadsheet

xenodium.com @alvaro — Fri, 18 Dec 2015 02:00:00 GMT

Examples from reference:

=GOOGLETRANSLATE("Hello World\n","en\n","es")
=GOOGLETRANSLATE(A2,B2,C2)
=GOOGLETRANSLATE(A2)

Organize your data with camlistore

xenodium.com @alvaro — Fri, 18 Dec 2015 02:00:00 GMT

Checking out camlistore to organize all sorts of data. Scaleway enables you to deploy camlistore servers.

Links - December 18th, 2015

Avy Faingezicht — Fri, 18 Dec 2015 02:00:00 GMT

Working with large files in Python

Umar Nizamani | RSS Feed — Thu, 17 Dec 2015 18:57:05 GMT

Recently I started on a pet project to explore a large data-set (32 million users) which was leaked to explore performance in Python. This…

Maps dev bookmarks

xenodium.com @alvaro — Thu, 17 Dec 2015 02:00:00 GMT

Use ImageMagick to convert image to grayscale

xenodium.com @alvaro — Thu, 17 Dec 2015 02:00:00 GMT

Another ImageMagick one-liner I'll likely forget.

mogrify -type Grayscale image.png

Links - December 17th, 2015

Avy Faingezicht — Thu, 17 Dec 2015 02:00:00 GMT

Ok, ok, I have a problem

infrequent oscillations — Thu, 17 Dec 2015 00:07:17 GMT

I already had a bunch of apps for making noises/musics but #november lead to - let’s be honest - a splurge. Which means it’s intervention time. The rule I’m not allowed to buy any more music/noise apps until I’ve made (and posted) something which each one. Since some of them are essentially filters or DAWs, combinations are allowed but should be minimal. Previously created content does not count for the purposes of this intervention.

Personally, I think we’re entering an era where acquisitions will be more common than IPOs.

🦉 olshansky 🦁 — Wed, 16 Dec 2015 02:01:46 GMT

The bubble is not bursting, but it's not growing at the same rate as it used to, and there's not much more air to fill it. VC money is…

Links - December 16th, 2015

Avy Faingezicht — Wed, 16 Dec 2015 02:00:00 GMT

Lolita

🦉 olshansky 🦁 — Wed, 16 Dec 2015 02:00:00 GMT

Olshansky's review of Lolita by Vladimir Nabokov

YouCompleteMe and CMake

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Tue, 15 Dec 2015 21:16:05 GMT

I recently updated my vim configuration to include the fine YouCompleteMe plugin for C++ development. Since the software project I am mostly working on during my PhD project uses CMake, configuring YCM turned out to require some additional steps.

First, I added

SET( CMAKE_EXPORT_COMPILE_COMMANDS ON )

to the main CMakeLists.txt file of our project. This ensures that CMake creates the file compile_commands.json in the build directory of the project. Next, we need CMake to provision the file properly:

IF( EXISTS "${CMAKE_CURRENT_BINARY_DIR}/compile_commands.json" )
  EXECUTE_PROCESS( COMMAND ${CMAKE_COMMAND} -E copy_if_different
    ${CMAKE_CURRENT_BINARY_DIR}/compile_commands.json
    ${CMAKE_CURRENT_SOURCE_DIR}/compile_commands.json
  )
ENDIF()

This places the file in the source directory, permitting us to place a single configuration file for YCM there. I have to add that I am not very happen with this solution yet. For one, this removes the ability to have a debug build in some location and a release build somewhere else because the build that is configured last gets to copy its compile commands—which of course tend to differ with different configuration options. Another reason why this solution is not entirely smart is that the copy is usually performed prior to CMake updating the compile_commands.json file. If you change many compile options for different files, you will thus have to manually run the cmake binary twice.

Even with all these quirks, it’s still worth it—YCM is just so amazing to have. Being a previous user of the QtCreator IDE, I am blown away by the speed with which the plugin manages to perform its duties.

As a last step, we need to place a .ycm_extra_conf.py configuration file in the project folder. By default, YCM will use a pre-defined list of compilation flags to apply to all files. This is of course insufficient for most larger projects, so we need to teach it to use the compile_commands.json instead.

This is easily achieved with the following configuration file:

import os
import ycm_core

from clang_helpers import PrepareClangFlags

def DirectoryOfThisScript():
    return os.path.dirname(os.path.abspath(__file__))

# This is the single most important line in this script. Everything else is just nice to have but
# not strictly necessary.
compilation_database_folder = DirectoryOfThisScript()

# This provides a safe fall-back if no compilation commands are available. You could also add a
# includes relative to your project directory, for example.
flags = [
    '-Wall',
    '-std=c++11',
    '-stdlib=libc++',
    '-x',
    'c++',
    '-I',
    '.',
    '-isystem', '/usr/local/include',
    '-isystem', '/usr/include',
    '-I.',
]

if compilation_database_folder:
    database = ycm_core.CompilationDatabase(compilation_database_folder)
else:
    database = None

SOURCE_EXTENSIONS = [ '.cpp', '.cxx', '.cc', '.c', '.m', '.mm' ]

def MakeRelativePathsInFlagsAbsolute( flags, working_directory ):
  if not working_directory:
    return list( flags )
  new_flags = []
  make_next_absolute = False
  path_flags = [ '-isystem', '-I', '-iquote', '--sysroot=' ]
  for flag in flags:
    new_flag = flag

    if make_next_absolute:
      make_next_absolute = False
      if not flag.startswith( '/' ):
        new_flag = os.path.join( working_directory, flag )

    for path_flag in path_flags:
      if flag == path_flag:
        make_next_absolute = True
        break

      if flag.startswith( path_flag ):
        path = flag[ len( path_flag ): ]
        new_flag = path_flag + os.path.join( working_directory, path )
        break

    if new_flag:
      new_flags.append( new_flag )
  return new_flags


def IsHeaderFile( filename ):
  extension = os.path.splitext( filename )[ 1 ]
  return extension in [ '.h', '.hxx', '.hpp', '.hh' ]


def GetCompilationInfoForFile( filename ):
  # The compilation_commands.json file generated by CMake does not have entries
  # for header files. So we do our best by asking the db for flags for a
  # corresponding source file, if any. If one exists, the flags for that file
  # should be good enough.
  if IsHeaderFile( filename ):
    basename = os.path.splitext( filename )[ 0 ]
    for extension in SOURCE_EXTENSIONS:
      replacement_file = basename + extension
      if os.path.exists( replacement_file ):
        compilation_info = database.GetCompilationInfoForFile(
          replacement_file )
        if compilation_info.compiler_flags_:
          return compilation_info
    return None
  return database.GetCompilationInfoForFile( filename )


def FlagsForFile( filename, **kwargs ):
  if database:
    # Bear in mind that compilation_info.compiler_flags_ does NOT return a
    # python list, but a "list-like" StringVec object
    compilation_info = GetCompilationInfoForFile( filename )
    if not compilation_info:
      return None

    final_flags = MakeRelativePathsInFlagsAbsolute(
      compilation_info.compiler_flags_,
      compilation_info.compiler_working_dir_ )

  else:
    relative_to = DirectoryOfThisScript()
    final_flags = MakeRelativePathsInFlagsAbsolute( flags, relative_to )

  return {
    'flags': final_flags,
    'do_cache': True
  }

The single most important line of this script is where you specify the compilation commands data base, compilation_database_folder = DirectoryOfThisScript(). Apart from the that, the script is a modified version of the default YCM configuration file, which the author strongly suggests to modify.

A last word of advice: Depending on your YCM configuration, you may want to add let g:ycm_extra_conf_globlist = [ '/path/to/your/project/*' ] to your .vimrc. This ensures that YCM will load the configuration file automatically and not let you confirm anything due to security reasons.

Happy code completing!

2016 Adventures

infrequent oscillations — Tue, 15 Dec 2015 15:29:00 GMT

Things for 2016 {Preliminary list - will repost on 2016-01-01} (Stretch goals / PBs) [Caveats / Requirements] “unsupported” means carrying everything from the start - no refuelling from shops. Only rest stops are allowed. Movementercisers Janathon Juneathon Survive a full 6 hour race [with at least 42km] (no walking) (“unsupported”) Run more than 45km (48km) (50km) (56km) (no walking) (“unsupported”) Finish an official marathon (5:15) (5:00) (4:45) Half PB (2:00) (1:55) (1:50) 10km PB (50:00) (47:30) (45:00) Walk BAR-KNG at least twice (9:45) (9:30) [at least once “unsupported”] Walk SGR-KNG (15:00) (“unsupported”) Run BAR-KNG (6:30) (6:00) 7 more parkruns (+3 new places) All of Regents Park 10km series RBR ranking under 16 (15) (14) Randomerisers inktober blogtober noisevember

Internet Protocol Security (IPSec)

Anardil — Tue, 15 Dec 2015 10:00:00 GMT

This document will describe the capabilities and mechanics of IPSec as outlined by the IETF

Links - December 15th, 2015

Avy Faingezicht — Tue, 15 Dec 2015 02:00:00 GMT

I have not actually read Feinstein's whole paper, but the abstract makes it seem like an amazing read: > In this paper we study the financial repercussions of the destruction of two fully armed and operational moon-sized battle stations (“Death Stars”) in a 4-year period and the dissolution of the galactic government in Star Wars. The emphasis of this work is to calibrate and simulate a model of the banking and financial systems within the galaxy. Along these lines, we measure the level of systemic risk that may have been generated by the death of Emperor Palpatine and the destruction of the second Death Star. We conclude by finding the economic resources the Rebel Alliance would need to have in reserve in order to prevent a financial crisis from gripping the galaxy through an optimally allocated banking bailout. Mind. Blown.

How to complain about Go

divan's blog — Mon, 14 Dec 2015 13:36:54 GMT

Over the years of existence of Go programming language, the articles with its critique was always popular, bringing a lot of discussion from both sides. Recently, Maksim Kochkin even created GitHub repo with curated list of articles complaining about golang’s imperfection.

So, is it true that ranting about Go flaws is a trend nowadays? With carefully gathered links in the repository above, we can check this! :) Unfortunately, there are only 17 articles in the list, which is a bit disappointing because it’s not enough for fine statistical analysis, but we can use this anyway.

Drill down Emacs dired with dired-subtree

xenodium.com @alvaro — Mon, 14 Dec 2015 02:00:00 GMT

JCS, from Irreal, recently highlighted fuco's dired-hacks. dired-subtree is super handy for drilling subdirectories down. Bound <tab> and <backtab> to toggle and cycle subtrees.

(use-package dired-subtree :ensure t
  :after dired
  :config
  (bind-key "<tab>" #'dired-subtree-toggle dired-mode-map)
  (bind-key "<backtab>" #'dired-subtree-cycle dired-mode-map))

GPG (GnuPG) examples

xenodium.com @alvaro — Mon, 14 Dec 2015 02:00:00 GMT

Generate key

gpg --full-generate-key

Export private key

gpg --export-secret-key -a <keyid> > <private.asc>

Import key

gpg --import < <private.asc>

Delete public key

gpg --delete-keys <keyid>

Delete private key

gpg --delete-secret-keys <keyid>

Edit key

gpg --edit-key <keyid>
gpg> uid (lists IDs)
gpg> uid 2 (marks ID)
gpg> deluid (deletes marked ID)
Really remove this user ID? (y/N) y

Change passphrase of the secret key

gpg --edit-key Your-Key-ID-Here
gpg> passwd
gpg> save

References

CSS bookmarks

xenodium.com @alvaro — Mon, 14 Dec 2015 02:00:00 GMT

Two Papers

Avy Faingezicht — Mon, 14 Dec 2015 02:00:00 GMT

This weekend I read two completely unrelated documents that have shaped, and in my opinion will continue to influence, the future of technology. First, I read the Twelve Factor App, by Adam Wiggins and the folks behind Heroku. Given that the first dynamic application I deployed online was a Ruby on Rails app on Heroku (thanks Michael Hartl!), a lot of the content seemed obvious to me. However after talking to some of my more experienced coworkers it was interesting to see how much of a paradigm shift cloud infrastructure has brought to the technology industry. This is reflected in company cultures, and in a way is discussed in the First Round Review article posted in today's list of links. Next, I read the Bitcoin White Paper by the infamous Satoshi Nakamoto. The paper was much more accessible than I expected. It is a very interesting read, and it gives a concice high level view of the inner workings of BTC. It is widely accepted that the release of this paper was a watershed moment in computer science and economics, as the mathematics backing the bitcoin network tackle big, previously unsolved, problems in distributed systems, such as the double spending problem and the byzantine generals problem. On the economics side, the technology allows its users to skirt trusted middlemen and third parties (i.e., banks and payment processors), lowering overall transaction costs. However, its effects go beyond changing cost structure. Bitcoin's controlled supply takes the public's expectations out of the inflation equation, by making credible claims on future currency issuance, preventing seignorage and possibly providing new tools to improve monetary policy. Additionally, the crytographical blockchain technology pioneered by bitcoin has also sparked innovation in other fields that benefit from the decentralization provided blockchain technologies: identity, settlements and smart contracts are only a few. Its really not clear how this will play out long term. After reading it, I have been playing with the idea of writing a simulation of the bitcoin network, stay tuned for that... The two documents could not have less in common, but I am certain that both will heavily impact business and technology for years to come. *Image: An afternoon at the Berlin Stock Exchange. H. Luders - Licensed under Public Domain via Wikimedia Commons_b_456.jpg)*

Links - December 14th, 2015

Avy Faingezicht — Mon, 14 Dec 2015 02:00:00 GMT

iSCSI SCSI-ID / Serial Persistence

smcleod.net — Mon, 14 Dec 2015 02:00:00 GMT

“Having a SCSI ID is a f*cking idiotic thing to do.”

- Linus Torvalds

…and after the amount of time I’ve wasted getting XenServer to play nicely with LIO iSCSI failover I tend to agree.

The Problem

One oddity of Xen / XenServer’s storage subsystem is that it identifies iSCSI storage repositories via a calculated SCSI ID rather than the iSCSI Serial - which would be the sane thing to do.

Chesterton's Fence

ho.dges.online — Mon, 14 Dec 2015 02:00:00 GMT

Chesterton’s fence is the principle that reforms should not be made until the reasoning behind the existing state of affairs is understood.
The quotation is from G. K. Chesterton’s 1929 book The Thing, in the chapter entitled “The Drift from Domesticity”:

“In the matter of reforming things, as distinct from deforming them, there is one plain and simple principle; a principle which will probably be called a paradox. There exists in such a case a certain institution or law; let us say, for the sake of simplicity, a fence or gate erected across a road. The more modern type of reformer goes gaily up to it and says, “I don’t see the use of this; let us clear it away.” To which the more intelligent type of reformer will do well to answer: “If you don’t see the use of it, I certainly won’t let you clear it away. Go away and think. Then, when you can come back and tell me that you do see the use of it, I may allow you to destroy it.”

Toward dynamic RADOS object class management

Noah Watkins — Mon, 14 Dec 2015 02:00:00 GMT

Persisting Lua-based RADOS object classes in Ceph.

RocketJump: The Show: Season 1

🦉 olshansky 🦁 — Sun, 13 Dec 2015 10:33:03 GMT

Olshansky's review of RocketJump: The Show: Season 1

The Dark Tower (The Dark Tower, #7)

Apurva Shukla's RSS Feed — Sun, 13 Dec 2015 10:00:00 GMT

⭐ ⭐ ⭐ ⭐ ⭐ A completely stunning ending to a sublimely fantastic epic saga! Completely loved it from the first book till the end, with King…

Musical Impostor: I made a thing. And I like it.

Joshua Rogers — Sat, 12 Dec 2015 14:00:00 GMT

I wrote a song this morning. I feel really good about it. I tend to record music, frown a little and then hide it far away from human ears. It isn't that I don't like it. I just don't want anyone else to realize how amateur it is and that I, by extension, must be an amateur. A fake. An imposter. So, I'm doing the imposter syndrome post? About time, I suppose.

Technology and Free Thought

Anardil — Sat, 12 Dec 2015 10:00:00 GMT

None

Resume partial downloads with ssh and rsync

xenodium.com @alvaro — Sat, 12 Dec 2015 02:00:00 GMT

rsync --rsync-path=/usr/local/bin/rsync \
      --partial \
      --progress \
      --rsh=ssh \
      john@host:/path/to/file \
      path/to/partial/file

Emacs text faces

xenodium.com @alvaro — Sat, 12 Dec 2015 02:00:00 GMT

Text faces = Text styles.
Face attributes: font, height, weight, slant, foreground/background color, and underlining or overlining.
Font lock mode automatically assigns faces to text.
M-x list-faces-display: Shows faces defined.
M-x helm-colors: Also handy.
Unspecified attributes are taken from 'default' face.

Simple ssh tunnel

xenodium.com @alvaro — Sat, 12 Dec 2015 02:00:00 GMT

Via @climagic, connections to tcp localhost:9909 will be made to 192.168.1.1:80 via SSH tunnel to home.

ssh -L 9909:192.168.1.1:80 home

Links - December 11th, 2015

Avy Faingezicht — Fri, 11 Dec 2015 02:00:00 GMT

Well, I finished the first season of Startup, and then, I got bored... Take note Alex Blumberg! The reasons why I was enjoying it so much were the SV outsider tone, the feeling of surprise when lingo had to be explained, and, perhaps most importantly, the fact that I listened to a whole season of Reply All before I started with StartUp. To me, Gimlet is just a more interesting business to learn about than Dating Ring (sorry matchmaker aficionados). Now, since I always _NeedMoreContent™_, I started a class on coursera and it has been awesome to learn about the low level pieces that make up a computer. It's been less than a week since I started, and I am already halfway through week 4 of the course. The class is aimed at people with 0 engineering background, and I would definitely recommend it if you are interested in computing.

painless publishing of android libraries

zserge's blog — Fri, 11 Dec 2015 02:00:00 GMT

Libraries are only good when it’s easy to share them. Android has done this wrong for a very long time. Node.js has npm and a new library can be published in a couple of minutes, usage is as simple as adding library name to package.json. Go has no package manager at all and publishing a library is as simple as “git push” and using is as simple as “import github.com/user/library”.

Spacemacs for atom - a few weeks later

David Mohl — Thu, 10 Dec 2015 13:59:23 GMT

I recently wrote about my attempt to bring spacemacs style editing to atom. A lot of stuff happened that I wanna tell you about!

In review

Since the last post, proton acquired 72 stars, 13 watches and 4 forks. We now have 3 people that actively contribute changes.

Proton became a good amount more stable, removed a lot of the ugly hacks it had in the beginning and added some really cool stuff on top. Let me give you a short overview:

We started creating a better default experience. The goal is that you just install proton, it reads your ~/.proton and bam! You have a superb editor experience. Since 0.1.2 we ship a better theme, a better font, better default settings and put a lot more power into the dotfile.
Step by step we are trying to allow users to toggle the entire atom experience from within ~/.proton or through the <spc> t menu. We created the base for pulling stuff like <spc> t n for relative line numbers off (which by the way is in a third party package and not part of atoms core) or <spc> t t to disable the tab-bar (which is a core package).
You can swap-out components more easily. Say you like nuclide and you want to use their quick-open tool instead of the default one. Now all you have to do is swap proton.core.quickOpenProvider to :nuclide and proton will replace it. Following this, adding different providers like swapping the file-tree with something else in the future will be no problem.
We have context aware shortcuts and modes on the way! This means that your <spc> menu could change depending on what file you are editing. <spc> m = in python could execute yapf while in javascript, it could use js-beautify. This would make proton a truly dynamic experience.
We added a handful new layers with more being added over time.

The crazy thing is that proton already bypassed sublimious in popularity, functionality and features.

Live reloading inside Atom

Thanks to the amazing thheller we now even have full support for hot code reloading while developing proton. It goes even this far that we might even be one of the onlys plugin out there that was able to pull this off with full REPL integration! Even the guys at the atom forums couldn’t do that 😁

With this new and even more stable build process, creating plugins in atom has never been this fun and easy! (ajom has been updated with the latest changes.)

The future

I am very impressed to see how proton is doing. As assumed, clojurescript is making atom such a powerful tool with a lot of unused potential. With modes-ish stuff on the way, we could even see something similar to emacs’s modes in the future. Just imagine that, composing modes inside atom! (Just wondering when the atom API will become limiting…)

Proton is on a steady way to become your central gateway when interacting with atom. Leveraging clojure goodies and now even support for live reloading and REPL development - developing a atom extension has never been this fun.

Now just to hope that more people adopt this and we might see more high quality and faster developed extensions on the way.

Spacemacs, we are coming for you!

Tutorial: Dealing with Disk Space Errors in Linux

Ben Overmyer's Site — Thu, 10 Dec 2015 02:00:00 GMT

If you're seeing errors on a server (say, npm complaining about an ENOSPC error, or apt failing to install something) that suggest the hard disk is full, use this guide to resolve the issue.

Troubleshooting

All of the following commands assume you are logged in as a sudo user on a Linux machine. Some of the commands, like df -hT, won't work exactly as advertised on something like, say, Solaris.

Checking disk space

First, check the amount of available disk space.

df -hT

If one of the available drives (such as /dev/xvda1) shows as 100% full, then you'll need to clear some space. The first place to look is /var/log/, especially if nginx is writing log files to disk. Clear out all old logs (anything with a .gz file extension is fair game for deletion). If that doesn't free up enough space, the next place to check is the application directory for whatever app is on the server. If there are more than 3 releases on the server, you can safely delete the oldest ones (until you have at least 2 remaining). You can also remove all the files in /tmp/, although that should be a last resort.

Checking inode usage

If you've confirmed that the hard drive isn't full (or cleared enough space that it isn't), and you're still seeing “disk is full” type errors, then you almost certainly have too few inodes available on the server. This means, basically, that there are too many files on the server.

Checking inode usage

Run the following command to check inode usage:

df -i

You'll see something like the following:

Filesystem     Inodes IUsed IFree IUse% Mounted on
udev             121K   409  120K    1% /dev
tmpfs            125K   479  124K    1% /run
/dev/vda1        1.9M  237K  1.7M   13% /
tmpfs            125K     2  125K    1% /dev/shm
tmpfs            125K     3  125K    1% /run/lock
tmpfs            125K    15  125K    1% /sys/fs/cgroup
tmpfs            125K     4  125K    1% /run/user/1000

The IUse% column is the one you're interested in. If IUse% is at or near 100%, then you have an inode usage problem.

Delete old kernels

Often, you can free up a ton of inodes if you delete old kernels. This is done with a pretty simple command:

sudo apt-get autoremove -y

Try that first. If that doesn't remove the old kernels, this page may help.

Script to check for directories with lots of files

This script will help find directories with large numbers of files. Put the following into a file list-files.sh in your home directory:

#!/bin/bash
# count_em - count files in all subdirectories under current directory.
echo 'echo $(ls -a "$1" | wc -l) $1' >/tmp/count_em_$
chmod 700 /tmp/count_em_$
find . -mount -type d -print0 | xargs -0 -n1 /tmp/count_em_$ | sort -n

Then make the file executable:

chmod +x ./list-files.sh

Finally, go to the root directory on the server and run the script:

cd /
~/list-files.sh

It may take a couple minutes to run, but it will present you with a list of all the directories with a number of files next to each, sorted in ascending order of number of files. Take a look at each of the high ones; that might give you an idea of what you need to clean up.

Some notes about good files to delete

Log files are safe to delete. So are old releases. Also check the /tmp/ directory; it can often be a hiding place for tons of files.

Disable OS X kernel_task throttling

David Schlachter — Wed, 09 Dec 2015 19:00:00 GMT

How to disable kernel_task CPU throttling in OS X. Tested on OS X El Capitan.

Adding a Play2 application as a sub-project to an existing sbt project

Home on The Log Book of Manuel Kießling — Wed, 09 Dec 2015 17:13:00 GMT

About this post Already working on an sbt-based Scala project with two sub-projects, I wanted to create a third sub-project, a Play2 application. The following describes what worked for me. Creating the sub-project structure Before adding the third sub-project, my sbt project structure looked like this: importer/ └ src/ ├ main/ └ test/ spark/ └ src/ ├ main/ └ test/ project/ └ plugins.sbt build.sbt Within build.

Things by Fleur Adcock

ho.dges.online — Wed, 09 Dec 2015 02:00:00 GMT

There are worse things than having behaved foolishly in public.
There are worse things than these miniature betrayals,
committed or endured or suspected; there are worse things
than not being able to sleep for thinking about them.
It is 5 a.m. All the worse things come stalking in
and stand icily about the bed looking worse and worse and worse.

Postgres and Node - Hands on using Postgres as a Document Store with MassiveJS

CRAIG KERSTIENS — Tue, 08 Dec 2015 22:55:56 GMT

JSONB in Postgres is absolutely awesome, but it’s taken a little while for libraries to come around to make it as useful as would be ideal. For those not following along with Postgres lately, here’s the quick catchup for it as a NoSQL database.

In Postgres 8.3 over 5 years ago Postgres received hstore a key/value store directly in Postgres. It’s big limitation was it was only for text
In the years after it got GIN and GiST indexes to make queries over hstore extremely fast indexing the entire collection
In Postgres 9.2 we got JSON… sort of. Really this way only text validation, but allowed us to create some functional indexes which were still nice.
In Postgres 9.4 we got JSONB - the B stands for Better according to @leinweber. Essentially this is a full binary JSON on disk, which can perform as fast as other NoSQL databases using JSON.

This is all great, but when it comes to using JSON you need a library that plays well here. As you might have guessed it from my previous post this is where MassiveJS comes in. Most ORMs take a more legacy approach to how they work with the database, in contrast the other side of the world believes in document only storage way is the future. In contrast Postgres believes there is a time and place for everything, just like Massive, except it believes Postgres is the path just as I do.

Alright, enough context, let’s take a look.

Getting all setup

First go ahead and create a database, let’s call it massive, and then let’s connect to it and create our example table:

$ createdb massive
$ psql massive
# create table posts (id serial primary key, body jsonb);

Now that we’ve got our database setup let’s seed it with some data. If you want you can simple hop over to the github repo and pull it down then run node load_json.js to load the example data. A quick look at it, given an example.json file we’re going to iterate over it. For each record in there, we’re going to call saveDoc. Based on our table which has a unique id key and a body jsonb field it’ll simply save our JSON document into that table:

var parsedJSON = require('./example.json');
for(i = 0; i < parsedJSON.posts.length; i++) {
db.saveDoc("posts", parsedJSON.posts[1], function(err,res){});
};

If you want to just take a look at this github repo, once you create a database you can run node load_json.js to seed it.

Why JSON at all?

JSON data is all over the place, in many cases it’s fast and flexible and allows you to move more quickly. Yes, much of the time normalizing your data can be useful, but there is something to be said for expediency saving some data and querying across it. Querying across some giant document also used to be much more expensive, but now with JSONB and it’s indexes that can be extremely fast.

Querying

So how do we go about querying? Well it’s pretty simple with Massive, they provide a nice findDoc function to let you just search for contents of a specific key within the document. Let’s say I wanted to pull back all posts that are in the Postgres category, well it’s as simple as:

db.posts.findDoc({title : 'Postgres'}, function(err,docs){
console.log(docs);
});

The real beauty of this is if you added a GIN index (which will index the entire document) this query will be quite performant.

Just make sure to add your GIN index:

CREATE INDEX idx_posts ON posts USING GIN(body jsonb_path_ops);
CREATE INDEX idx_posts_search ON posts USING GIN(search);

But even better, with Massive it’ll automatically add these for you if you just start saving docs. It will automatically create the table and appropriate indexes, just doing the correct thing out of the box.

Full text and JSON

Cool, so you can do an exact look up. Which is great when you’re matching a category… which could be easily normalized. It’s great when you’re matching numbers, which also could likely reside in their own column. But what about when you’re searching over a large document, or a set of keys within some document which may require several joins, or indeterminate data structure, well you may want to search for the presence of that string at all. As you may have guessed this is quite trivial.

db.posts.searchDoc({
keys : ["title", "category"],
term : ["Postgres"]
}, function(err,docs){
console.log(docs);
})

Hopefully it’s pretty straight forward, but to be very clear. Call out the document table you want to search, then the keys you’ll want to include in the search, then the term. This will search for any place the contents that string are found in matching values for those keys.

Which will nicely yield the expected documents:

[ { link: 'http://www.craigkerstiens.com/2015/05/08/upsert-lands-in-postgres-9.5/',
title: 'Upsert Lands in PostgreSQL 9.5 – a First Look',
category: 'Postgres',
comments: [ [Object] ],
id: 2 },
{ link: 'http://www.craigkerstiens.com/2015/11/30/massive-node-postgres-an-overview-and-intro/',
title: 'Node, Postgres, MassiveJS - a Better Database Experience',
id: 3 } ]

In conclusion

While Massive isn’t perfect, its approach to storing queries in files, using the schema vs. having to define your models in code and the database, and it’s smooth document integration makes it a real contender as a better database library when working with Node. Give it a try and let me know your thoughts.

Apple Pay

infrequent oscillations — Tue, 08 Dec 2015 14:03:28 GMT

http://time.com/money/4139330/apple-pay-unused/ Survey: Only 20% of those that can use Apple Pay have used it Trustev surveyed 1,000 consumers who owned Apple Pay-compatible iPhones about their Apple Pay habits. (The iPhone 6, iPhone 6 Plus, iPhone 6s, and iPhone 6s Plus support Apple Pay, while older iPhones do not.) It found that about 80% had not used Apple Pay even once. Of the 20% who had, most (56%) only used it about once a week.

Preview HTML pages on github

xenodium.com @alvaro — Tue, 08 Dec 2015 02:00:00 GMT

Prepend with http://htmlpreview.github.io/?. For example: http://htmlpreview.github.io/?https://github.com/xenodium/xenodium.github.io/blob/master/index.html

Integration testing in Go using Docker

divan's blog — Mon, 07 Dec 2015 17:36:54 GMT

Note: this post was originally written for the Go Advent 2015 series, but I discovered that a post with almost exactly the same subject (and even similar code!) already planned :) That’s amazing.

Golang is often used for writing microservices and various backends. Often these type of software do some computation, read/write data on external storage and expose it’s API via http handlers. All this functionality is remarkably easy to implement in Go and, especially if you’re creating 12factor-compatible app, Go is your friend here.

Upgrading a Symfony project from 2.6 to 3.0

Home on The Log Book of Manuel Kießling — Mon, 07 Dec 2015 17:13:00 GMT

About this post Symfony 3.0 has been released recently, and I wanted to upgrade my side project JourneyMonitor, which was still based on Symfony 2.6, as quickly as possible. This post explains how I approached the upgrade, and shows which parts of the project structure and code had to be changed in order to get the project working again. Preparing the upgrade The code base for the project – the CONTROL component of the JourneyMonitor project – lives at https://github.

#noisevember

infrequent oscillations — Mon, 07 Dec 2015 14:50:26 GMT

The output In reverse order; unless otherwise indicated, things were created on iOS and recorded over Audiobus to Audioshare before exporting to Soundcloud. https://soundcloud.com/zimpenfish/tidal-noisevember (Tidal through Audacity) https://soundcloud.com/zimpenfish/much-indulgent-very-soundscape (SoundScaper) https://soundcloud.com/zimpenfish/just-beatwave-drums (Beatwave) https://soundcloud.com/zimpenfish/coconut-latte (Figure); https://soundcloud.com/zimpenfish/finsbury-park-bonus (Figure) https://soundcloud.com/zimpenfish/beyond-mere-indulgence (SoundScaper) https://soundcloud.com/zimpenfish/more-self-indulgence (FieldScaper) created from https://soundcloud.com/zimpenfish/before-self-indulgence (PixiTracker) https://soundcloud.com/zimpenfish/i-honestly-have-no-idea (Fieldscaper) https://soundcloud.com/zimpenfish/bass-player-mangled (Fieldscaper, creating from a recording of someone practicing a bass riff in the office kitchen) https://soundcloud.com/zimpenfish/mildly-delayed (SoundScaper, using one of the Bitwiz library tracks) https://soundcloud.

Are recent trends in company IPOs signaling an imminent recession?

🦉 olshansky 🦁 — Mon, 07 Dec 2015 04:37:02 GMT

Commodities are tanking, the fed is still contemplating an increase in interest rates, and several countries are either in or on the verge…

Flutter setup

xenodium.com @alvaro — Mon, 07 Dec 2015 02:00:00 GMT

Based on Getting Started with Flutter.

$ curl -O https://storage.googleapis.com/dart-archive/channels/stable/release/1.13.0/sdk/dartsdk-macos-x64-release.zip
$ unzip dartsdk-macos-x64-release.zip
$ export PATH=`pwd`/dart-sdk/bin:$PATH

Verify with:

$ pub --version

Playing with Dart's analysis server

xenodium.com @alvaro — Mon, 07 Dec 2015 02:00:00 GMT

Dart SDK ships with an analysis server. Very handy if you'd like to write a completion plugin for your favorite editor. The API is well documented. Of interest, there's LocalDartServer.java, part of dartedit.

$ dart path/to/bin/snapshots/analysis_server.dart.snapshot  --sdk=path/to/dart-sdk

NOTE: The server reads requests from standard input. Either escape or execute the following as one-liner json requests.

{
  "id": "1\n",
  "method": "analysis.setAnalysisRoots\n",
  "params": {
    "included": [
      "path/to/your/dart/project"
    ],
    "excluded": []
  }
}

{
  "id": "3\n",
  "method": "completion.getSuggestions\n",
  "params": {
    "file": "path/to/some/file.dart\n",
    "offset": 673
  }
}

Dart bookmarks

xenodium.com @alvaro — Mon, 07 Dec 2015 02:00:00 GMT

Command Line: Newbie to Ninja

Pete's Points — Sun, 06 Dec 2015 19:52:52 GMT

Part One: Bow to Your Sensei

These tips are the dojo ground rules you must learn before even the most basic command line work can get done. If you haven't got these down, everything is going to take much longer than it needs to. These are also definitely a prerequisite to any command line activity while pair programming if you don't want your pair to pull their hair out during the session.

tab completion

The shell can help you out with typing long filesystem paths and avoiding typos in command names. It does this with a feature called "tab completion". Tab completion allows you to start entering just the first few characters of something then type the <TAB> key and the shell will complete it for you. So for example, instead of typing cd ./external/plugins, you could type cd ./ex<TAB> and the shell would expand the rest of the directory name "external", allowing you to keep typing pl<TAB> to expand plugins. This saves a lot of typing and prevents typos from ruining your day. Pretty much every experienced command line user I've paired with uses tab completion on nearly every command they run.

navigating command history

Another glaring sign that it's your first day at the command line is re-typing a command you recently ran. The shell keeps a history of commands you ran and makes it easy to re-run a previous command because this is something that happens all the time in typical interactive shell usage. Here are some shell history basics that will keep you moving fast.

Use the up and down arrow keys to scroll back/forward through your history
- CTRL-p (previous) and CTRL-n (next) do the same thing and touch typists may prefer these as your hands don't need to leave the home row

And for something you remember running but it was far enough back that you'd need to hit the up arrow many times to find it, there is CTRL-r for reverse interactive searching your history. Type CTRL-r then any portion of the command you remember and the shell will search backward through your history for a match. Keep typing to find the exact match and hit ENTER when you see it to re-run that command. If you need to edit it before running again to modify it slightly, use the arrow keys to move your cursor and the line will be loaded as your current command but not yet executed.

For example:

cd /tmp
sudo restart nginx
ls /var/log
vi /tmp/some/file.txt
<CTRL-r>restart<ENTER> #<- quickly re-run the sudo restart nginx command

line editing keyboard shortcuts

Second only to re-typing previous commands in terms of obvious beginner indicators would be backspacing over most of a command to make an edit to the beginning of the command, then re-typing everything you deleted.

Instead, use the shell's interactive line editing features. These are keyboard shortcuts taken from the 2 most popular command-line text editors: vi and emacs. Most shells support both of these. By default the bash shell loads the emacs keybindings. You can switch to vi bindings with set -o vi and back to emacs with set -o emacs.

The most important ones for emacs are CTRL-a to move the cursor to the beginning of the line and CTRL-e to go to the end.

If you use vi for text editing, you'll love having all your favorite move/edit commands available.

don't cd for one-off commands

Another common pattern I see with beginners at the command line is a tendency to cd to a new directory for every command. Think of cd like taking off your coat and staying a while. If you are just running a single command dealing with a different directory, there's no need to cd to that directory first. cd when you are going to stay in a single directory and run a bunch of commands for a while. This will keep your paths short since you can use relative paths and tab completion. But if you just want to create a directory in /tmp, just run mkdir /tmp/foo without cd /tmp first.

In particular, avoid the tendency to cd every time you use the ls command. It usually ends up causing you extra work to cd back to your project directory. ls can take the names of directories/files to list as command line arguments so instead of

cd /var/log
ls

just run

ls /var/log

understand space-delimited args

"Do I need to quote it?" is something beginners often ask. You may have heard that all the complex edge cases of bash quoting rules are complex and tricky, and that is indeed true. Here be dragons. However, for the basic cases, it's really simple. By default the command you enter is separated into distinct values on spaces. So if you enter ls /tmp /home/me /var/log the shell is going to parse that as 4 distinct tokens with ls being the program to run and each of the 3 directory paths as the arguments. So what that means is when a value you want to pass to a program contains a space, you need to quote it so the shell knows it is one value not several. So if I have a file path with a space in it like /tmp/uprade notes.txt and I want to pass that to the wc program, I need to type wc "/tmp/upgrade notes.txt" so the wc program gets just 1 argument that is a valid filesystem path instead of 2 arguments: /tmp/upgrade and notes.txt, neither of which are valid. This rule applies to the shell entirely, and it doesn't matter which program is at the start of your command line command, the shell does the parsing before passing the values as arguments to the program being executed.

Part Two: Breaking Boards

Now that you've got the basics, let's add some intermediate skills and understanding that will make you more effective.

Let's start with 2 issues that are a constant nuisance to beginners but once properly understood, immediately solvable every time for the intermediate developer.

no such file or directory

One of the core subsystem of every computer is a hierarchical data structure called the filesystem. Overall, it's pretty great. It's extremely general purpose and versatile while being pretty straightforward. However, it is pretty easy to get confused about filesystem paths and inadvertently ask the computer to read a file path that does not actually exist. Sadly, the common pattern for this is the bad path gets passed in from a high level application and the actual error doesn't occur until deep in the core of the OS where no context exists about who or what wants to read this file and for what purpose. So at the command line, the error message can often be entirely unhelpful "No such file or directory" with an unfamiliar path you've never heard of. This can be quite confusing and frustrating.

However, once you understand the basics of the filesystem including importantly absolute paths, relative paths, ./, ../, symlinks, and the per-process current working directory, these errors suddenly all become trivial to solve.

It's worth spending a few hours mastering this stuff so you can handle it properly once and for all.

learn how command lines are verbalized

When discussing things in person, on the phone, or over video chat, it helps to have a common understanding of how command line commands are verbalized so your coworker doesn't have to give you keystroke-by-keystroke instructions like your keyboard is the control panel of a Boeing 787.

Pair up with someone and practice speaking command lines to each other and learn to parse what is a command name, which is a command line option, and how some of these odd unix/C abbreviations are pronounced.

Here's a few examples to get you started:

"make dir dash P temp foo" = mkdir -p /tmp/foo
"L S temp foo" = ls /tmp/foo
"vee eye etsy profile" = vi /etc/profile
"find dot dash name star dot text pipe to W C dash L" = find . -name '*.txt' | wc -l

Part Three: Enter the Dragon

Here are a few more advanced techniques.

learn when you don't need sudo

sudo is a command that comes with some legacy baggage about what it is, why it is needed, and when to use it. I commonly see beginners try it at random when things don't work. Don't do this. sudo let's you run an individual command with full root permissions. You need it when doing sysadmin type things like installing software system-wide, applying security patches, editing core OS configuration files, starting and stopping system services etc. You should NOT need it for doing normal user things like running read-only commands, managing files within your home directory, running programs that are already installed, etc. Read through an introduction to sudo and get clear on excatly when it is or is not necessary.

OSX pbpaste/pbcopy

If you work on OSX for development, the pbpaste and pbcopy commands are convenient bridges from the main OS clipboard ("pasteboard" is what the "pb" stands for) and the command line. For example, if you copy a bunch of text in your editor and want to see how many characters there are, you can run pbpaste | wc -c on the command line. Similarly, if you want to copy a program's output you can pipe data into pbcopy. For example, I might run something like npm run lint | pbcopy if I know that is going to print out a bunch of errors that I want to work through one at a time. I'll then paste it into an empty buffer in my text editor so I can fix each item then delete it's line from my editor buffer until I'm done.

use short-lived functions and aliases

Many advanced command line users learn to effectively define aliases and shell functions in their dotfiles to improve their productivity. This is a great idea and a programmer's dotfiles can serve as an essential tool as well as a source of pride and the occasional geek contest about whose dotfiles are more sophisticated/ridiculous.

However, one practice I see less often but is also really useful is defining quick aliases or functions just for the current session. I often do this if I know I'll need to run a series of commands over and over in a sequence like edit some code, stop a service, copy new code into place, restart a service. I might quickly define an alias for all those steps like alias r="cp ./src /deploy && stop myapp && start myapp". This lets me focus on a rapid turnaround in my edit/test cycle and makes me more effective.

The conventional name for the repo where your dotfiles live on github is just "dotfiles" and by searching around you can find and study how others use the shell. It feels a little bit like getting a tour of someone's workshop and you can readily see people's styles and preferences. Perhaps the most famous repo is Mathias Bynens dotfiles with over 11K stars on github.

use long options in shell scripts for readability

When typing interactive commands, brevity is good for speed and convenience. However, when you are writing a script that is getting checked into source code management as part of a project, consider using the long options. This will make it easier to read the script and understand what it does for the maintainers, who are unlikely to be as intimately familiar with every command your script runs as you were when you wrote it.

So instead of:

curl -b 'active=1' -D /tmp/foo \
  -e http://example.com -O http://example.com

Code it as:

curl \
  --cookie 'active=1' \
  --dump-header /tmp/headers.txt \
  --referer http://example.com \
  --remote-name \
  http://example.com

The longer names will tend to be easier to understand without referring to the documentation. The backslash-continued lines also help with readability.

references and further reading

My dotfiles on github. Feel free to study and mine for snippets.
Tim Pope's dotfiles. Maybe yours would look like this if you had written a hundred vim plugins
Github dotfiles repos sorted by most stars
Bash Emacs Editing Mode Cheat Sheet by Peteris Krumins who writes an outstanding blog with incredible unix expertise
The Unofficial Bash Strict Mode
Jessica Dillon's great command line tutorial series on the Quick Left Blog
Shell Check shell script linter
Bash Automated Testing System by Sam Stephenson
Oh My Zsh Popular plugin system for customizing zsh

Study Notes In The Information Age

Beetle Space — Sun, 06 Dec 2015 10:00:00 GMT

It has been a long time since I was a college student. In those days I took all my notes in the classroom with a notebook. One notebook for each course.

Looking back at the experience, this had some down sides:

Some of what I wrote down was “useless”. Things …

Flutter bookmarks

xenodium.com @alvaro — Sun, 06 Dec 2015 02:00:00 GMT

Swift bookmarks

xenodium.com @alvaro — Sun, 06 Dec 2015 02:00:00 GMT

Load Lua RADOS classes from local file system

Noah Watkins — Sun, 06 Dec 2015 02:00:00 GMT

Supporting the installation of Lua-based RADOS object classes.

Overlay Network Performance of Docker

Mustafa Akın — Sat, 05 Dec 2015 02:00:00 GMT

Since Docker 1.9, the multi-host networks can be used very easily. All you have to do is just setup your Swarm cluster and use docker network create -d mynet and voila: Your multi host network is ready. I want to do a benchmark of this new feature, however, the official multi host networking examples are in Virtualbox and local performance can be misleading. I created an image called mustafaakin/alpine-iperf just has the Alpine image plus iperf network benchmarking tool installed.

World of Commodore conference

Jason Eckert's Website and Blog — Sat, 05 Dec 2015 02:00:00 GMT

This is the first year that I’ve been to the World of Commodore event in Mississauga, Ontario. It’s a yearly gathering of Commodorks who create new software and hardware for vintage Commodore computers. I had no idea that it even existed until someone I knew told me about it. Since it’s relatively close to me, I just had to go. Plus, I have many Commodore computers in my vintage computer collection.

[Computing] TLS and the Policy MitM Armageddon

devever.net/~hl — Thu, 03 Dec 2015 02:00:00 GMT

Kazakhstan has announced that it will be performing Man-in-the-Middle attacks on all HTTPS traffic from the beginning of 2016. Users in Kazakhstan will be forced to install the CA certificate of a rogue CA in order to allow their traffic to be intercepted.

Creating a horizontal comparison bar chart using Tableau

Christo’s Blog — Thu, 03 Dec 2015 01:02:38 GMT

I have put together a screencast with some step by step instructions on how to create a horizontal comparison bar chart in Tableau.

Snow Leopard Server with Mavericks clients, and why to avoid Mac Server

Jonathan Chang — Tue, 01 Dec 2015 08:54:08 GMT

Note: the below text was written in 2014 but never posted. I have kept the same tense since I don’t want to rewrite it.

Our lab has a setup where we have multiple workstation iMacs that connect to a Mac Mini server, with network accounts stored on the server. Previously, both server and clients were running Snow Leopard, but I’ve just upgraded two of the clients to Mavericks while keeping the server on Snow Leopard. (Upgrading to Mavericks server costs money). I’ve been running this setup for a few weeks now and nothing major seems to have broken. For details of our lab’s setup, and why you should avoid Mac Server, read on…

In 2011 my advisor purchased an expensive Mac Mini Snow Leopard server for the lab, to store data generated by the lab and do other things that only servers can do. The lab had several iMacs that served as workstations for lab members, as well as a Mac Pro used for data analysis. Data were originally stored on the Mac Pro as well, but with the new server we could separate those tasks and use the Mac Pro only for analysis work.

At the time, I didn’t do too much research into Mac Server and assumed that things would “just work” out of the box. I also figured that the best way to handle the “data storage” aspect was to store everything on the server, including user accounts. In essence, this means that user home directories were stored server-side and all home folder access and user authentication is done over the network.

The network account system has a couple of advantages. Every member of the lab could have their own account, files, and settings, that would work regardless of which computer they signed in on. It’s actually really cool to be able to just sign out of one computer and then sit down at another and keep going right where you left off. Central control of user data and user accounts also meant that backups were also centralized, hopefully insuring against data being lost because it was on an old machine that was rarely used.

However, the drawbacks are pretty significant. The major one is that the server is a single point of failure for every single person. Originally, if the file server was down, you couldn’t get access to the shared lab data, but your own data were still intact on your workstation. Now, if either the server or the network were down, you couldn’t login to your own computer, never mind access your files.

Other quality of life issues include:

Time Machine backups won’t work on the server due to bad interactions with Open Directory databases. We use rsnapshot to the other internal drive and CCC to an external drive.
Home directories served over AFP prevent multiple accounts being signed into the same workstation. This is because the first user signed in will mount the AFP share under /Users with their own permissions, so the second user will try to mount the same share, fail due to permission issues, and throw a cryptic error message confusing everyone.
Home directories served under NFS don’t play nice with many programs, especially ones that assume that all user directories on Macs is stored under /Users. I also encountered a bug with Dropbox not working since it was trying to acquire a file lock and failing (probably because Apple’s NFS implementation was bad, or perhaps that NFS in general is bad). To Dropbox’s credit, I emailed their support team and they eventually fixed the bug.
Every client computer’s DNS must point to the server computer. This actually isn’t necessary if your upstream DNS assigns a fixed hostname for your server. Ours does, but the problem is that I originally configured the server to have a different hostname than its actual name. Once past the initial setup, changing the server’s name is so onerous that most guides I consulted recommended doing a clean install of Mac Server with the correct hostname. By running your own DNS and pointing all client computers to your server, the server can lie to itself and its connecting clients about its own name and thus everyone is happy. Note that it’s actually possible to change the hostname: during initial setup I made a typo in one of the boxes that you must enter the hostname into and in doing so created a difficult to track down bug that caused client computers to sometimes stall during login for minutes at a time. Tracking down this one involved looking at a lot of log files and noticing attempts to connect to an incorrectly-spelled host. I fixed this one by looking through all the configuration files for the misspelled name and correcting it.
You will be terrified of restarting the server for updates. Every time I have updated the server, it refuses to work correctly in spectacular and exciting new ways. This one has probably shortened my life expectancy by a couple of years, at least.

Update 2015: Basically, I don’t think using OS X server is Worth It. Thankfully Apple seems to have scaled back their server offerings so hopefully no one else is falling into this trap. Our new setup uses a dedicated network appliance (Synology NAS) to serve files, combined with several (quite beefy) shared Linux workstations. All devices can connect to the Synology NAS over NFS. The issue of having shared desktops but personal accounts is no longer a problem, since all analyses are performed on the workstations, and everyone has their own personal laptop that they used to remotely login to the shared workstations. Although this is less secure in the sense that it’s possible for people to trample over each other’s files, in practice this hasn’t really been that big of an issue. There are separate horrors associated with accessing an NFS share on Linux systems with automount, but I’ll save that for a separate post.

Denim Tablet Case

june.kim — Tue, 01 Dec 2015 02:00:00 GMT

Links - December 1st, 2015

Avy Faingezicht — Tue, 01 Dec 2015 02:00:00 GMT

I have been listening to Startup the past couple of days, so I have not been reading that much lately. It provides a view of the tech and startup world from an outsider's perspective, and when discussing concepts that I take as general knowledge it constantly makes me think "Wait, most people don't know that?" If you have not listened to it, you should.

Node, Postgres, MassiveJS - A better database experience

CRAIG KERSTIENS — Mon, 30 Nov 2015 22:55:56 GMT

First some background–I’ve always had a bit of a love hate relationship with ORMs. ORMs are great for basic crud applications, which inevitably happens at some point for an app. The main two problems I have with ORMs is:

They treat all databases as equal (yes, this is a little overgeneralized but typically true). They claim to do this for database portability, but in reality an app still can’t just up and move from one to another.
They don’t handle complex queries well at all. As someone that sees SQL as a very powerful language, taking away all the power just leaves me with pain.

Of course these aren’t the only issues with them, just the two ones I personally run into over and over.

In some playing with Node I was optimistic to explore Massive.JS as it seems to buck the trend of just imitating all other ORMs. My initial results–it makes me want to do more with Node just for this library. After all the power of a language is the ecosystem of libraries around it, not just the core language. So let’s take a quick tour through with a few highlights of what makes it really great.

Getting setup

Without further adieu here’s a quick tour around it.

First let’s pull down the example database from PostgresGuide

Then let’s setup out Node app:

$ npm init
$ npm install massive --save

Connecting and querying

Now let’s try to connect and say query a user from within our database. Create the following as an index.js file, then run with node index.js:

var massive = require("massive");
var connectionString = "postgres://ckerstiens:@localhost/example";
var db = massive.connectSync({connectionString : connectionString});
db.users.find(1, function(err,res){
console.log(res);
});

Upon first run if you’re like me and use the PostgresGuide example database (which I now need to go back and tidy up), you’ll get the following:

db.users.find(1, function(err,res){
^
TypeError: Cannot read property 'find' of undefined

I can’t describe how awesome it is to see this. What’s happening is when Massive loads up it’s connecting to your database, checking what tables you have. In this case though because we don’t have a proper primary key defined it doesn’t load them. It could treat id as some magical field of course like Rails used to and ignore the need for an index, but instead it not only encourages a good database design but requires it.

So let’s go back and create our index in our database:

$ psql example
$ alter table users add primary key (id);

Alright now let’s run our script again with node index.js and see what we have:

{ id: 1,
email: 'john.doe@gmail.com',
created_at: Thu Sep 24 2015 03:42:52 GMT-0700 (PDT),
deleted_at: null }

Perfect! Now we’re all connected and it even queried our database for us. Now let’s take a few more look at some of the operators.

Running an arbitrary query

db.run will let me run any arbritrary SQL. An example such as db.run("select 'hello'") will produce [ { ‘?column?’: ‘hello’ } ].

This makes it nice and easier for us to break out of the standard ORM model and just run SQL.

Find for quick look ups

Similar to so many other database tools find will offer you the most common quick look ups:

$ db.users.find({email: 'jane.doe@gmail.com'}, function(err, res){console.log(res)});
$ db.users.find({'created_at >': '2015-09-24'}, function(err, res){console.log(res)});

And of course there’s a where operator for multiple conditions.

Structuring queries in your application

While in the next post I’ll dig in deep to JSON, this is perhaps my favorite feature of Massive… It’s design for pulling out queries into individudal SQL files. Simply create a db folder and put your SQL in there. Let’s take the most basic example of our user email lookup and put it in user_lookup.sql

SELECT *
FROM users
WHERE email = $1

Now back in our application we can run this and pass in a parameter to it:

db.user_lookup(['jane.doe@gmail.com'], function(err,res){
console.log(res);
});

This separation of our queries from our code makes it easier to track them, view diffs, and even more so create very readable SQL.

Up next

So sure, you can connect to a database, you can query some things. There were a couple of small but more novel things that we blew through in here. First is the fact I didn’t have to define all my schema, it just knew it as it really should. The separation of SQL queries you’ll custom write into files is simple, but will make for much more maintainable applications over the long term. And best of all is the JSON support, which I’ll get to soon…

We Are Your Friends

🦉 olshansky 🦁 — Mon, 30 Nov 2015 09:30:25 GMT

Olshansky's review of We Are Your Friends

RAM is the new disk – and how to measure its performance – Part 3 – CPU Instructions & Cycles

Tanel Poder Blog — Mon, 30 Nov 2015 08:45:58 GMT

If you haven’t read the previous parts of this series yet, here are the links: [ Part 1 | Part 2 ].

A Refresher

In the first part of this series I said that RAM access is the slow component of a modern in-memory database engine and for performance you’d want to reduce RAM access as much as possible. Reduced memory traffic thanks to the new columnar data formats is the most important enabler for the awesome In-Memory processing performance and SIMD is just icing on the cake.

In the second part I also showed how to measure the CPU efficiency of your (Oracle) process using a Linux perf stat command. How well your applications actually utilize your CPU execution units depends on many factors. The biggest factor is your process’es cache efficiency that depends on the CPU cache size and your application’s memory access patterns. Regardless of what the OS CPU accounting tools like top or vmstat may show you, your “100% busy” CPUs may actually spend a significant amount of their cycles internally idle, with a stalled pipeline, waiting for some event (like a memory line arrival from RAM) to happen.

Luckily there are plenty of tools for measuring what’s actually going on inside the CPUs, thanks to modern processors having CPU Performance Counters (CPC) built in to them.

A key derived metric for understanding CPU-efficiency is the IPC (instructions per cycle). Years ago people were actually talking about the inverse metric CPI (cycles per instruction) as on average it took more than one CPU cycle to complete an instruction’s execution (again, due to the abovementioned reasons like memory stalls). However, thanks to today’s superscalar processors with out-of-order execution on a modern CPU’s multiple execution units – and with large CPU caches – a well-optimized application can execute multiple instructions per a single CPU cycle, thus it’s more natural to use the IPC (instructions-per-cycle) metric. With IPC, higher is better.

Installing Emacs spaceline

xenodium.com @alvaro — Mon, 30 Nov 2015 02:00:00 GMT

Gave Spaceline a try. Spacemacs's powerline theme. Setup was super simple (Thanks Eivind Fonn and Sylvain Benner):

(use-package spaceline :ensure t
  :config
  (use-package spaceline-config
    :config
    (spaceline-toggle-minor-modes-off)
    (spaceline-toggle-buffer-encoding-off)
    (spaceline-toggle-buffer-encoding-abbrev-off)
    (setq powerline-default-separator 'rounded)
    (setq spaceline-highlight-face-func 'spaceline-highlight-face-evil-state)
    (spaceline-define-segment line-column
      "The current line and column numbers."
      "l:%l c:%2c")
    (spaceline-define-segment time
      "The current time."
      (format-time-string "%H:%M"))
    (spaceline-define-segment date
      "The current date."
      (format-time-string "%h %d"))
    (spaceline-toggle-time-on)
    (spaceline-emacs-theme 'date 'time))

Links - November 30, 2015

Avy Faingezicht — Mon, 30 Nov 2015 02:00:00 GMT

Took a break for thanksgiving. Read a lot. Choosing was hard. This week's winner seems to be Noah Smith.

Quick update, and Hugo

Ben Overmyer's Site — Mon, 30 Nov 2015 02:00:00 GMT

Well, it's been about two months since my last post, which ironically was about my intention to write more frequently. To be fair, I wrote in my journal a number of times between then and now, but this is my first public post since then. Today I discovered a Go-based static site generator called Hugo that fascinates me with its speed and usability. I'm going to try using it to replace the current structure and content of benovermyer.com.

Codemotion 2015

GeekWare - Daniel Pecos Martínez — Sun, 29 Nov 2015 23:00:00 GMT

Day after Codemotion 2015, this is my overall opinion of the event: meh. Don’t get me wrong, technical sessions have been quite interesting, full of really nice people and organization must have worked really to get the event running. But, in my opinion, this event is dying of success. Way too many assistants, making really hard to walk from one session to another (even having 15 min between them), or having to arrive 30 min before start if you’re intention was to have a nice place to plug your laptop and be able to see the presentation.

Now, speaking about content, it’s really nice to have so much diversity to choose from in every time slot (although sometimes you were force to discard really nice sessions). Personally, I chose to assist to an JVM / Scala track that covered approximately the whole time slots of the first day (and part of the second) without significant overlaps. BTW, really interesting stuff is happening around the JVM, nothing to envy to the Node.js community.

But I also wanted to have a look into different technologies which were newer to me, such as Docker or Big Data. And it’s impressive how much has the technology evolved and what amazing things are people achieving in these fields. Currently, recorded sessions are ~~not yet published~~ already published!, but I’m pretty sure it will be worth taking a look into what I missed.

But as always, the best thing of these events is the people you meet, old and new acquaintances. And, of course, nothing as a good as an old arcade machine and a StreetFighter match in the chill out area to relax and have some fun 😉

Entwicklung und Betrieb einer Symfony2 Webanwendung – Teil 1

Home on The Log Book of Manuel Kießling — Sun, 29 Nov 2015 17:13:00 GMT

Über diesen Artikel Dies ist ein Crosspost vom GALERIA Kaufhof Technology Blog. Vor kurzem standen wir vor der Herausforderung, eine kleine Onlineanwendung für eine zeitlich begrenzte Rabattaktion zu realisieren, die keinerlei Verbindung mit dem Galeria.de Webshop hatte. Während der Technologiestack rund um unseren Onlineshop auf Scala, Ruby und Casssandra basiert, wurde hier die Entscheidung gefällt, die Anwendung außerhalb unserer bestehenden Dienste und Systeme zu realisieren, und auch nicht im Kontext unserer Scala und Ruby Teams, mit dem Ziel den normalen Produktentwicklungsprozess nicht mit diesem Sonderprojekt zu “stören”.

package.el incomprehensible buffer

xenodium.com @alvaro — Sun, 29 Nov 2015 02:00:00 GMT

Came across "incomprehensible buffer" error in package.el. Workaround patch:

--- a/lisp/emacs-lisp/package.el
+++ b/lisp/emacs-lisp/package.el
@@ -1161,6 +1161,7 @@ package--with-work-buffer
(let* ((url (concat ,url-1 ,file))
       (callback (lambda (status)
                   (let ((b (current-buffer)))
+                    (goto-char (point-min))
                     (unwind-protect (wrap-errors
                                      (when-let ((er (plist-get
                                                      status :error)))
                                        (error "Error retrieving: %s %S" url er))

Leading bookmarks

xenodium.com @alvaro — Sun, 29 Nov 2015 02:00:00 GMT

Online reading backlog

xenodium.com @alvaro — Sun, 29 Nov 2015 02:00:00 GMT

[TODO]{.todo .TODO} Phrack 69.

[TODO]{.todo .TODO} A Simple Formula for Changing Our Behavior.

[TODO]{.todo .TODO} Be Grateful More Often.

[TODO]{.todo .TODO} GTD sucks for creative work.

[TODO]{.todo .TODO} Land, Capital, Attention: This Time it Is the Same.

[TODO]{.todo .TODO} Mindset: What You Believe Affects What You Achieve (Gates Notes).

[TODO]{.todo .TODO} The Case for Getting Rid of Borders—Completely.

[TODO]{.todo .TODO} The Ultimate Guide to Personal Productivity Methods.

[TODO]{.todo .TODO} Thing Explainer: A Basic Guide for Curious Minds (Gates Notes).

[TODO]{.todo .TODO} Your body language shapes who you are.

Jurassic World

🦉 olshansky 🦁 — Sat, 28 Nov 2015 15:21:53 GMT

Olshansky's review of Jurassic World

Travel lifestyle bookmarks

xenodium.com @alvaro — Sat, 28 Nov 2015 02:00:00 GMT

Documentation has a Cost and a Value

The Cranky Developer on Crater Moon Development — Sat, 28 Nov 2015 02:00:00 GMT

The cost of documentation is proportional to change frequency and distance from the code

Spacemacs for Atom

David Mohl — Fri, 27 Nov 2015 20:24:23 GMT

If you read my last posts you might notice that I really like editors. Something about finding better and better tools is very interesting and fun. Plus using a new tool for a couple of weeks is keeping things fresh even when you work on the same codebase.

The problem is that since I started using spacemacs, I can’t really use anything else. It just… doesn’t work. The shortcuts are too awkward, get overriden by third party plugins and all the manual config orchestration is just a pain.

When I went back to Sublime Text for a little, I ended up writing sublimious, a plugin which ports a big chunk of the spacemacs toolset into Sublime Text:

It works really REALLY good! But eventually I ran into problems implementing more things. Stuff like relative line numbers is close to impossible to do without a big performance hit or awkward behavior. Or the gutter still only being able to hold 1 element in it. What’s up with that?

Performance wise it is crazy fast but let’s face it: Sublime Text is dead even if the community doesn’t want to admit it. Waiting until John Skinner to come along and commit a new feature once a year is just taking too long.

My first time with Atom?

Now I am a little bit biased towards atom. I looked at it multiple times but it felt too slow, too in-performant and most of the plugins were just trash-level (sorry if I insulted some of my readers but they are just… not good!).

All the problems I have about plugins overwriting keybindings are here stronger than anywhere else. A plugin can literally change every little aspect of the editor. This is a really great thing, don’t get me wrong! But at the same time you need to take double care of what you are installing.

Github also did a lot of weird design decicions that I can’t follow. Here are a few:

Emoji as commit messages

I tried to nibble my way through the codebase and recent changes a bit but emojis… man. I mean, it’s cool that you have fun with it but for outsiders it’s unnecessary confusing. 🎨 is probably style related, I get that, but what the heck is 📝? 🔥?

Plus using emojis is not a substitute for a commit message. It’s like me commiting with the word “style.”

Coffeescript

I used to like coffeescript a lot but reading through the atom sourcecode made me realise why I will probably not touch it again.

Take this snippet for example:

@eventElement.dispatchEvent(new CustomEvent(name, bubbles: true, cancelable: true))

What is it doing? Why do some of the arguments have colons? Where does the object start and where does it end? The constructor is getting 3 arguments, right? Right?

I literally had to run this through a coffeescript parser to understand what it’s doing:

this.eventElement.dispatchEvent(
  new CustomEvent(name, {
    bubbles: true,
    cancelable: true,
  }),
);

confusing objects everywhere

I’m a big functional guy and everywhere I go in the atom sourcecode, I am dealing with objects after objects. I can’t count how many @element assignments and reads I saw. Some things are getting assigned at the top of an object, used at the bottom and changed 3x until it reached that spot.

cson

For some reason someone decided that coffeescript is so great that we should also start writing json in it! Because… commas are evil or something?

I quickly ran away from it

Atom, meet clojurescript

Now, since then and now I got a lot better in clojurescript so I decided to give it another go. I created a little lein template for creating atom plugins in clojurescript and hacked a bit here and there.

Turns out this works really well! Clojurescript and atom are like… emacs and elisp! You have all the power of clojurescript combined with all the customisation options atom gives you. You still have to deal with some quirks like passing objects around but still - this is making the idea of having a central configuration system so much more possible!

Our clojurescript plugin could literally change how every other package reacts: how it is set up, what keybindings it uses, what options it needs and more. We could even change how other plugins look inside the editor by manipulating it’s DOM representation.

Short said, in the last few days I started porting a bit of functionality from sublimious to atom.

For sublimious, I did a lot of hacking. I basically collected all the configuration and flushed it into real files that are then getting re-loaded by sublimetext. For atom, I don’t need to do that.

How do I set a editor config? Like this!

(defn set-config! [selector value]
  (.set config selector value))

How do I wipe the existing user config so it doesn’t conflict with our new one? One line.

(map atom-env/unset-config! (atom-env/get-all-settings)))

The only hack I had to implement is the ability to install packages from within a package. That’s currently not possible. Here’s how I did it anyway:

(defn install-package [package-name]
  (if (is-installed? package-name)
    true
    (try
      (do
        (.execSync child-process (str (get-apm-path) " install " package-name " --no-colors"))
        true)
      (catch js/Error e
        false))))

(I’m just calling apm through node!)

Is it ready?

Not even close to but I am already a lot more happy with it than with sublimious before. It feels more… powerful! Maybe cljs+atom can really be a alternative to emacs at some point. Let me dream 😊

In the meantime, if you want to help, please check it out here.

Good Parenting, or High-Tech Cruises?

Welcome to Marco Fioretti's website! on Stop at Zona-M — Fri, 27 Nov 2015 17:57:39 GMT

Some months ago, TechCrunch reported that, on High-Tech cruise ships like the Quantum Of The Sea, “Wi-Fi is fast and it’s everywhere” for one specific reason that I really don’t like .

Great piece!

🦉 olshansky 🦁 — Fri, 27 Nov 2015 04:04:54 GMT

My takeaway was that as long all your needs are met, money just acts as insurance. Nothing changes when you get home / auto / health…

The Nature of Order

june.kim — Fri, 27 Nov 2015 02:00:00 GMT

Notes on the Synthesis of Form

june.kim — Fri, 27 Nov 2015 02:00:00 GMT

The Timeless Way of Building

june.kim — Fri, 27 Nov 2015 02:00:00 GMT

SoundCloud RSS Feeds

Caius Theory — Fri, 27 Nov 2015 01:33:37 GMT

SoundCloud appears to have gained popularity in recent times for hosting podcasts on. As a consumer of their service they’re pretty good at everything except having a visible RSS feed on a profile page for a show! If I want to listen to a show in my podcast app of choice, an RSS feed is the easiest way for me to achieve that.

Turns out SoundCloud do have RSS feeds, they’re just well hidden and unfindable from the profile page itself. Thankfully, you can construct the URL for it from information on the profile page, and here’s a bookmarklet that will do it for you:

javascript:var%20userURI;var%20metaTags=document.getElementsByTagName(
%22meta%22);for(var%20i=0;i<metaTags.length;i++){t=metaTags[i];if(
t.attributes[%22property%22]&&t.attributes[%22property%22].value==
%22al:ios:url%22){userURI=t.content;}}if(userURI){u=userURI.split(%22//%22)[1];
window.location=%22http://feeds.soundcloud.com/users/soundcloud:%22+u+
%22/sounds.rss%22;}

Or as a handy link to copy to your bookmarks bar. Simply click/run that when on a SoundCloud profile page and you’ll be taken to the RSS Feed URL.

SQL bookmarks

xenodium.com @alvaro — Thu, 26 Nov 2015 02:00:00 GMT

Unix/Linux tools bookmarks

xenodium.com @alvaro — Thu, 26 Nov 2015 02:00:00 GMT

Couchbase React Native bookmarks

xenodium.com @alvaro — Thu, 26 Nov 2015 02:00:00 GMT

Installing Emacs 25 devel on Mac OS X

xenodium.com @alvaro — Thu, 26 Nov 2015 02:00:00 GMT

Stable

brew update
brew install emacs --HEAD --use-git-head --with-cocoa --with-srgb --with-gnutls
brew linkapps emacs

Development

  brew update
  brew install emacs --devel --with-cocoa --with-srgb --with-gnutls
  brew linkapps emacs
then

Had problems loading seq. Removed byte-compiled packages:

$ find ~/.emacs.d/elpa -iname *.elc -exec rm '{}' \;

Terrible Customer Support from Chase

Pete's Points — Thu, 26 Nov 2015 01:49:10 GMT

So the Chase mobile banking app does not work properly on my new Google Nexus 5X phone. When taking a photo to deposit a check, the on-screen preview is rotated 180 degrees. This makes it really awkward to try to center the check because when you move the phone intuitively to fix alignment, it gets worse not better.

I googled this and found it's a known problem due to some technical details about this particular phone and it requires the chase mobile app developers to code a fix and release an update via the google play store.

Here's a reddit thread where the Tech Lead for the Android Camera Framework confirms it.

Here's another reddit thread where 2 other users confirm the problem in the Chase app specifically.

Here's an android central thread on this topic.

So I tried to tell Chase about it. It went like this:

Sent them a message via their "Secure Messaging Center" within their banking web site
- They came back saying "we checked, it's not an actual problem. Fuck off."
I sent them feedback via the google play store which amounts basically to an email
- They responded "psych! Yeah we can't actually communicate over these messages, sent us a message via our online banking web site"
When I pressed again in their online banking message system that yes this is a real issue, here are the links they said "Oh fuck off we can't actually do tech support here, call this telephone number, I'm sure it will be fun to speak web URLs to them letter-by-letter"
So I called them and spoke with tech support who thought I was saying a particular photograph was upside down for a while. When I finally explained this is a bug in their app to his satisfaction, he started to tell me to call some other number...

Then I got profane, which I almost never do, but I said something along the lines of "No. Fuck that. I've already sent you emails in 2 different systems and called one telephone number. I'm not sending smoke signals. I'm not telegraphing anyone. You fucking record this bug in your system and communicate it internally to the right people on my behalf". The profanity seemed to actually prompt helpfulness, which is a sad truth, but eventually I found a reddit shortlink and spoke it to him over the phone. Hopefully they'll fix their app.

Multi-variable To-do List

David Schlachter — Wed, 25 Nov 2015 19:00:00 GMT

To-do list weighing both the deadline and the value of the task. Helpful for balancing urgency and importance, tuned for university course work and exams.

Macro-writing Macros

John Jacobsen — Wed, 25 Nov 2015 02:00:00 GMT

… in which we explore the power of macros, and macro-writing macros, to DRY out repetitive code.

I’ve been writing Clojure code full time for nearly two years now. I have a pretty good feel for the language, its virtues and its faults. Mostly, I appreciate its virtues (though I still wish the REPL started faster).

For me one of the language’s attractions has always been that it’s a Lisp — a “homoiconic” language, i.e., one defined in terms of its own data structures. Homoiconicity has one primary virtue, which is that it makes metaprogramming more powerful and straightforward than it is in non-homoiconic languages (arguably at some cost to readability).

In Lisp, this metaprogramming is accomplished with macros, which are functions that transform your code during a separate stage of compilation. In other words, you write little programs to change your programs before they execute. In effect, you extend the compiler itself.

I run a Clojure study group at work and find that it can be hard to explain the utility (or appeal) of this to newcomers to Lisp. This is partly because macros do things you can’t easily do in other languages, and because the things you want to do tend to relate to abstractions latent in a particular codebase.

While playing around with 3d rendering in Quil, I recently came across a use case that reminded me of the following quote by Paul Graham:

The shape of a program should reflect only the problem it needs to solve. Any other regularity in the code is a sign, to me at least, that I’m using abstractions that aren’t powerful enough— often that I’m generating by hand the expansions of some macro that I need to write¹.

In Quil, there are multiple situations in which one needs to create a temporary context to carry out a series of operations, restoring the original state afterwards:

Save current style with push-style; change style and draw stuff; restore previous style with pop-style.
Start shape with begin-shape; draw vertices; end-shape to end.
Save current position/rotation with push-matrix; translate / rotate and draw stuff; restore old position/rotation with pop-matrix.

Here’s an example:

(push-matrix)
(try
  (push-style)
  (try
    (fill 255)
    (no-stroke)
    (translate [10 10 10])
    (begin-shape)
    (try
      (vertex x1 y1 0)
      (vertex x2 y2 0)
      (vertex x2 y2 h)
      (vertex x1 y1 h)
      (vertex x1 y1 0)
      (finally
        (end-shape)))
    (finally
      (pop-style)))
  (finally
    (pop-matrix)))

The (try ... (finally ...)) constructions may not be strictly needed for a Quil drawing, but it’s a good habit to guarantee that stateful context changes are undone, even if problems occur.

In a complex Quil drawing the idioms for saving style, translation state, and denoting shapes appear often enough that one hungers for a more compact way of representing each. Here’s one way to do it:

(defmacro with-style [& body]
  `(do
     (push-style)
     (try
        ~@body
       (finally
         (pop-style)))))

(defmacro with-matrix [& body]
  `(do
     (push-matrix)
     (try
        ~@body
       (finally
         (pop-matrix)))))

(defmacro with-shape [& body]
  `(do
     (begin-shape)
     (try
        ~@body
       (finally
         (end-shape)))))

The original code then becomes more compact and easier to read:

(with-matrix
  (with-style
    (fill 255)
    (no-stroke)
    (translate [10 10 10])
    (with-shape
      (vertex x1 y1 0)
      (vertex x2 y2 0)
      (vertex x2 y2 h)
      (vertex x1 y1 h)
      (vertex x1 y1 0))))

In this example code, the contexts with-matrix, etc. appear so often that the resulting savings in lines of code and mental overhead for the reader is substantial.

However, the astute reader will realize that the macro definitions themselves are pretty repetitive—in fact, they look almost identical except for the setup and teardown details (this kind of “context manager” pattern is common enough that Python has its own language construct for it).

I generally reach for macros when I have a pattern that occurs with obvious repetition that’s not easy to abstract out using just pure functions. Control abstractions such as loops or exception handling are common examples. (I find this situation occurs especially frequently when writing test code).

In any case, the solution for our repetitive macros could be something like:

(defmacro defcontext
  [nom setup teardown]
  `(defmacro ~(symbol (str “with-” nom))
     [~'& body#]
     `(do
        ~'~setup
        (try
          ~@body#
          (finally
            ~'~teardown)))))

Yikes! I have to admit I had to write a lot of macros, and also refer to this helpful page for reference, before I could write (and grok) this macro.

With defcontext in hand, our repetitive macro code just becomes:

(defcontext style (push-style) (pop-style))
(defcontext shape (begin-shape) (end-shape))
(defcontext matrix (push-matrix) (pop-matrix))

These are exactly equivalent to the three context macros (with-*) defined above.

With a little effort, it’s actually not too hard to construct such a nested macro. It’s largely a matter of writing out the code you want to generate, and then writing the code that generates it, testing with macroexpand-1 at the REPL as you go. This page by A. Malloy has a lot of helpful remarks, including this cautionary note: “Think twice before trying to nest macros: it’s usually the wrong answer.” In this case, I actually think it’s the right answer, because the pattern of a context with setup and teardown is so common that I know I’ll reuse this macro for many other things—we have effectively added one of my favorite Python features to Clojure in just a few lines of code[fn::To be even more like Python’s context managers, defcontext would want to enable the user to bind some local state resulting from the setup phase of execution (“ with x() as y: ” idiom); examples include file descriptors or database connections. This is left as an exercise for the reader.]

There’s a saying in the Clojure community: data > functions > macros. I’m a big believer in this. Clojure’s powerful built-in abstractions for wrangling data in all its forms make it the language I prefer above all others these days. But occasionally that means wrangling the data that is the code itself, thereby reaping the benefits in power, brevity and expressiveness.

Image generated by the Quil code used for this example.

Paul Graham, "Revenge of the Nerds": http://www.paulgraham.com/icad.html

Diagram tools bookmarks

xenodium.com @alvaro — Wed, 25 Nov 2015 02:00:00 GMT

My New Youtube Channel

Tanel Poder Blog — Tue, 24 Nov 2015 06:30:28 GMT

I have created a new youtube channel – and have uploaded some videos there already! Bookmark & Subscribe here:

https://www.youtube.com/tanelpoder

More stuff is coming over the next weeks & months :-)

Licensing bookmarks

xenodium.com @alvaro — Tue, 24 Nov 2015 02:00:00 GMT

choosealicense.com (Choosing an open source license doesn’t need to be scary).
choosealicense.com (github).
tldrlegal.com (Software Licenses in Plain English).

[Computing] Why the AGPL is often unenforceable

devever.net/~hl — Tue, 24 Nov 2015 02:00:00 GMT

The AGPL is intended to require that where the source code of open source network service software is modified, that code is made available even if the software is not distributed.

[Computing] Linux's GPLv2 licence is routinely violated

devever.net/~hl — Tue, 24 Nov 2015 02:00:00 GMT

The Linux kernel is licenced under the GPLv2. There are several notable things about this:

Synology bookmarks

xenodium.com @alvaro — Mon, 23 Nov 2015 02:00:00 GMT

The Circle (The Circle, #1)

🦉 olshansky 🦁 — Mon, 23 Nov 2015 02:00:00 GMT

Olshansky's review of The Circle (The Circle, #1) by Dave Eggers

Backup bookmarks

xenodium.com @alvaro — Sun, 22 Nov 2015 02:00:00 GMT

Making hummus

xenodium.com @alvaro — Sun, 22 Nov 2015 02:00:00 GMT

Made hummus, based on Delicious Istanbul's 5 Secrets to Perfect Hummus (wayback machine) post.

160 g dry chickpeas.
4 cloves garlic minced.
1 tsp fine sea salt.
1 1/2 tbsp lemon juice.
1/4 tsp ground cumin.
6 tbsp tahini paste.
2/3 cup cooking water.
Extra virgin olive oil, for serving.
Red pepper flakes, for serving.
Zahter mixture, for serving.

Soak chickpeas overnight.
Discard water and rinse chickpeas.
Cook in low heat (about 5 cups water) for 1.5 hours for until soft (but keeping shape.) Check if can be mashed with thumnb.
Save cooking water.
Peal chickpeas (optional).
Blend ingredients until silky paste. Taste and add lemon/salt/cooking water.

Keeps in fridge for 3-4 days. Freeze otherwise.

Nara travel bookmarks

xenodium.com @alvaro — Sun, 22 Nov 2015 02:00:00 GMT

Late Fragment by Raymond Carver

ho.dges.online — Sun, 22 Nov 2015 02:00:00 GMT

And did you get what
you wanted from this life, even so?
I did.
And what did you want?
To call myself beloved, to feel myself
beloved on the earth.

Lenin On History

ho.dges.online — Sun, 22 Nov 2015 02:00:00 GMT

“There are decades where nothing happens; and there are weeks where decades happen.”

Kubernetes bookmarks

xenodium.com @alvaro — Sat, 21 Nov 2015 02:00:00 GMT

Docker bookmarks

xenodium.com @alvaro — Sat, 21 Nov 2015 02:00:00 GMT

[[https://news.ycombinator.com/item?id=25619319

][Dockerfile Best Practices | Hacker News]].

Angular bookmarks

xenodium.com @alvaro — Sat, 21 Nov 2015 02:00:00 GMT

Egghead.io's Angular 2 lessons.

Mac OS bookmarks

xenodium.com @alvaro — Sat, 21 Nov 2015 02:00:00 GMT

easy_install-\>pip-\>conda

xenodium.com @alvaro — Sat, 21 Nov 2015 02:00:00 GMT

Spotted Conda package manager. It handles python installations, in addition to package management. There's also a package index provided by Binstar. Installed Miniconda, the bare bones Conda environment.

Can't find a python package in Binstar? Here's a post on Using PyPi Packages with Conda. If that fails, you can try pip from your Conda python environment.

Links - November 20, 2015

Avy Faingezicht — Fri, 20 Nov 2015 02:00:00 GMT

Build vs. Buy, Part 5

The Cranky Developer on Crater Moon Development — Fri, 20 Nov 2015 02:00:00 GMT

Buy for parity, build for advantage

[Computing] Insyde BIOS Issues

devever.net/~hl — Fri, 20 Nov 2015 02:00:00 GMT

Insyde is a relatively new UEFI-style BIOS manufacturer. Unfortunately their product, Insyde H2O, leaves a great deal to be desired in terms of code quality.

The Walking Dead: Season 6

🦉 olshansky 🦁 — Thu, 19 Nov 2015 14:30:57 GMT

Olshansky's review of The Walking Dead: Season 6

Demand driven iOS apps with om and react native

David Mohl — Thu, 19 Nov 2015 13:01:23 GMT

I recently talked a lot about om and react native. Since people seemed quite interested in this topic, I decided to hold a little presentation at the last Tokyo iOS meetup in November.

People were a bit overwhelmed (many never saw clojurescript before) but the general feedback was quite good. I am sure I convinced at least 1 person to give it a spin!

In any way, here is my talk from that iOS meetup. Slides are available here.

This was also the first time we decided to record things so excuse the mediocore quality. I experimented by putting a external microphone on the table next to me but didn’t realise that the macbook fan blowed straight into it. Still slightly better than the in-camera mic but still room for improvements.

I for my part am very excited about clojurescript on native and super happy to be part of this still young but growing community. Clojurescript will come for you - if you want it or not. Better start learning it now! 😈

White Chocolate Protein Flapjacks

infrequent oscillations — Thu, 19 Nov 2015 09:45:25 GMT

Almost entirely a recipe I got from @chrismusther. Specific ingredients I used linked in [brackets]. Ingredients 75g Scottish Oats [Sainsbury’s] 75g Flax (Milled) [Linwoods Organic] 120g Whey Protein [Precision Engineered, Banana] 1 Apple (large) [Sainsbury’s 5 for £1] 300ml Apple Juice [Sainsbury’s Basics] 50g Mixed Fruit [Sainsbury’s Basics Dried] 30g Morello Cherries [Sainsbury’s Taste The Difference - Alternative] 20g Currants [Whitworths Juicy] 100g White Chocolate Chips [Sainsbury’s Cook’s] Method Put the oats, flax, whey protein, and chocolate chips into a bowl and mix loosely by hand to ensure a diverse community Grease up your silicon baking tray - 8" square is my preference Preheat your oven to about 150º / 305ºF / Gas Mark 2 Chop the apple as finely as you want the bits in your flapjacks but no finer than your sanity can allow Put the apple chunks, apple juice, and various fruits into a pan - precise ordering is irrelevant here Apply strong heat until the apple chunks go soft but before the juice really starts evaporating Pour the hot steaming fruit cocktail onto the aforementioned diverse oaty-flaxy-chocolatey-protein community and stir slowly but thoroughly This will create an astonishingly sticky mess that smells amazing - RESIST ALL TEMPTATION TO MASH YOUR FACE IN IT Carefully spoon the sticky mess into your baking tray, evening it out as much as you can (good luck with that) Place baking tray in oven and relax for 25-30 minutes until the top is firm Remove your delicious goodness from the oven and leave to cool before enjoying with a nice cup of tea Nutrition Based on the specific ingredients I used, MyFitnessPal reckons a 1/16th slice contains -

Build vs. Buy, Part 4

The Cranky Developer on Crater Moon Development — Thu, 19 Nov 2015 02:00:00 GMT

Customization and programmability of commerical and bespoke software

Why does the theatre cost ALL MY MONEY?

infrequent oscillations — Wed, 18 Nov 2015 14:26:08 GMT

In which we learn about the cost of theatre productions.

Paper Lantern

june.kim — Wed, 18 Nov 2015 02:00:00 GMT

Links - November 18, 2015

Avy Faingezicht — Wed, 18 Nov 2015 02:00:00 GMT

Build Vs. Buy, Part 3

The Cranky Developer on Crater Moon Development — Wed, 18 Nov 2015 02:00:00 GMT

Polist and quality comparisons in vendor vs in-house software

Organizational Patterns of Agile Software Development

june.kim — Tue, 17 Nov 2015 02:00:00 GMT

Links - November 17, 2015

Avy Faingezicht — Tue, 17 Nov 2015 02:00:00 GMT

Intro to FlightJS

Dylan Paulus' Blog — Tue, 17 Nov 2015 02:00:00 GMT

This post is part of the archive. These are mainly random posts that were written throughout the years in no coherent order.

FlightJS

FlightJs isn’t the new kid on the block. This little framework has gone unnoticed in Github-land for years. Taken from the FlightJS Github: “Flight is a lightweight, component-based JavaScript framework that maps behavior to DOM nodes.” Reworded, we use reusable components that listen for event calls. These events then in turn perform actions on the DOM or manipulate pieces of our code. Today we will get started with FlightJs, and show how simple it is to use. Though this is an introduction guide, I will assume that we have a fair proficiency with HTML, CSS, and JavaScript.
Getting Started

In this Intro to FlightJS guide we will do what many intro framework guides do, make a counter. Let’s get to work.
Head over to the FlightJS Github, and install FlightJS using your favorite method.
HTML code:

<h1>FlightJS Counter</h1>
<div class="”counter”">
  <h2 class="”counter-total”">0</h2>
  <div class="”counter-function”">
    <a href="”#subtract”" class="”counter-subtract”">-</a>
    <a href="”#add”" class="”counter-add”">+</a>
  </div>
</div>

Counter

To get started, let us create a FlightJS component. Since we will be creating a counter, we’ll create a Counter component.

var Counter = flight.component(function() {
   this.defaultAttrs({});
   this.after(“initialize”, function() {
   });
});

We did a few things in the example above. First, we created a new Flight component called Counter. Next, we defined that we have no default attributes to the Counter component (this will change soon). Attributes provide two big benefits: Component attributes create private variables for a component.

Secondly, attributes give us a way to assign unique data to each instance of a component. Lastly, we have this weird this.after(“initialize”) thing going on at the end. This is where a lot of the power comes from. Flight will execute the function we provide, as the second argument, after the initialize function has ran (initialize is an internal flight function) . Basically we’re telling Flight that once it gets itself in order, let us do what we need to.

So what does a counter need? We need a total number that will be counted on. Oh yeah, and some buttons, that when clicked, either add or subtract one from the total. Let’s add this to our JavaScript.

var Counter = flight.component(function() {
   this.defaultAttrs({
      "subtractSelector": ".counter-subtract",
      "additionSelector": ".counter-add",
      "count": 0
   });
   this.subtract(function() {});
   this.add(function() {});
   this.after(“initialize”, function() {
     this.on("click", {
        "subtractSelector": this.subtract,
        "additionSelector": this.add
     });
   });
});

First we added some attributes. These attributes provide us a DOM element for our subtract and add buttons—named subtractSelector and additionSelector respectively—and defines our starting number count. We then created some stubbed functions for once the selector’s event is triggered. Finally, an event handler is created. this.on(“”) creates an event binding it to whatever we define within the quotes. In this case, we are waiting on a click event to be performed on our DOM selectors subtractSelector and additionSelector. Once a click is recognized, the corresponding function is called. (this.subtract() or this.add())

Continuing, here is our finished code:

var Counter = flight.component(function() {
   this.defaultAttrs({
      "subtractSelector": ".counter-subtract",
      "additionSelector": ".counter-add",
      "count": 0
   });
   this.subtract = function() {
      this.attr.count--;
   }

   this.add = function() {
      this.attr.count++;
   }

   this.uiDisplayCount = function() {
      this.$node.find(“h2.counter-total”).html( this.attr.count );
   }

   this.after("add subtract", function() {
      this.uiDisplayCount();
   });
   this.after(“initialize”, function() {
     this.uiDisplayCount();
     this.on("click", {
        "subtractSelector": this.subtract,
        "additionSelector": this.add
     });
   });
});

Finally we get functionality. The subtract and add function perform a -1 or +1 on the total count depending on which event is triggered. Once we perform these actions on our count, we have no way of displaying the new information. This is where this.uiDisplayCount plays its role. From any jQuery enthusiast, this function is straight forward. Take our count attribute, and insert it into the html of .counter-total. But, what is this.$node? This.$node is a property given to us from Flight that grabs a jQuery object of the DOM this component is attached to. Later on we will attach the Counter component to the DOM, and see how this plays out. Going to the next function, we tell our component to execute the uiDisplayCount function after the add or subtract functions are done being executed.

Now there is one thing missing, how do we actually attach this to the DOM?
Counter.attachTo(“.counter”, { count: 100 });

Going back to before, our this.$node property would return a jQuery object of our .counter, $(“.counter”). As we can see by the second parameter, we can also change what our default attributes are once we attach the component. This helps enforce the modular code Flight preaches. We can change the attributes to fit whatever the situation.

Counter.attachTo(“.counter”, { count: 100 });
Counter.attachTo(“.counter-new”, {
                   count: 0,
                   subtractSelector: ".new-subtract"
                });
Counter.attachTo(".counter-new-new", {
                   subtractSelector: ".s",
                   additionSelector: ".a"
                });

We Did It

And that is it, we completed our counter! We can see from this little example, Flight gives us some powerful tools in creating modulated code without the need to learn new paradigms. As always, there is much more Flight can do. For additional information on Flight check out their Github.

Counter Example

Here is our finished product: http://codepen.io/ganderzz/pen/pjNdNq

Build Vs. Buy, Part 2.

The Cranky Developer on Crater Moon Development — Tue, 17 Nov 2015 02:00:00 GMT

Upgrades and feature creep in commercial software

It Is Not The Critic Who Counts by Theodore Roosevelt

ho.dges.online — Tue, 17 Nov 2015 02:00:00 GMT

It is not the critic who counts. Not the man who points out how the strong man stumbles or where the diet of deeds could have done them better. The credit belongs to the man who is actually in the arena, whose face is marred by dust and sweat and blood. Who strives valiantly. Who errs. Who comes short again and again. Because there is no effort without error and shortcoming.

Announcing White Glove

Pete's Points — Mon, 16 Nov 2015 17:20:59 GMT

I've open-sourced my mongodb/couchdb data integrity analysis tool called white glove. You can find it at focusaurus/white-glove on github and white-glove on npm. It helps with both schema discovery of a new or unfamiliar database as well as scanning your data for inconsistencies that may help you prevent problems before they cause any impact to end users.

Elite Proxies

Anardil — Mon, 16 Nov 2015 10:00:00 GMT

What they are, how they work, how they can be useful, and how you can use one yourself

Ant-Man

🦉 olshansky 🦁 — Mon, 16 Nov 2015 09:49:24 GMT

Olshansky's review of Ant-Man

Traditional music bookmarks

xenodium.com @alvaro — Mon, 16 Nov 2015 02:00:00 GMT

Links - November 16, 2015

Avy Faingezicht — Mon, 16 Nov 2015 02:00:00 GMT

Grounding, part 4: Explaining leaky abstractions

D13V — Mon, 16 Nov 2015 02:00:00 GMT

Some concepts are very complex and need a long explanation. But we might decide to take a shortcut and simplify the concept when describing it.

That short explanation is misleading.

Lets see why… For example, we might be trying to explain electro-magnetism. A full explanation will include Maxwell’s equations, definitions of electric and magnetic fields a discussion of ferromagnets, and more. A shortcut explanation might be that magnets attract like rubber bands.

The shortcut might suffice for a while. It gets the rough idea of “attraction” across, but cannot be trusted as a base for further inference. Rubber bands break if they are stretched too much, but magnets still work over very far distances - albeit weakly. Coldness increases the power of magnets, but decreases the flexibility of a rubber band.

To refine our understanding of magnets, we need to start imagining rubber bands that behave differently with temperature, knowing that they get weaker with heath. This is an exception to our common sense, and it requires a bunch of effort to remember it and apply it properly.

Pretty soon we start keeping all these exceptions to the rule in our heads. This is wasteful, and defeats the point of the short explanation. We could have redirected that extra effort to towards learning Maxwell’s equations properly. A person which blindly follows the rubber band model might save effort for a little bit, but over time they will spend more mental energy to keep their mental model in sync with reality. They will have larger error and disappointment when the model fails…

The cautious thing would be to realize when the model works and when it doesn’t work and to only use it, when it works, for a while, and not depend too much on it. Misleading models are a shaky foundation for reasoning.

But which models are not misleading? Even the laws of physics get disputed by other physicists.

The answer is that there are no perfect models. Every model is an abstraction over reality, and all abstractions are leaky. That’s the law of leaky abstractions, defined by Joel Spolsky. All models that we use in the daily life are imprecise in some way.

But lets not despair. Some models are more precise and trustworthy than others. They can go a very, very long way towards practicality.

Physics models, for example, a.k.a. the laws of physics, are usually so precise that the scientists in CERN can use them to take single particles and hit them with really high speed, and measure very tiny deviations from their expectations.

Other models, usually the non-scientific ones, are much less reliable. Many economic models are only approximate, and they have been derived in situations, that are no longer applicable as the markets and regulations evolve. Lets just remember this every time there is a market crash.

To be fair, even leak-free abstractions might not be able to fully predict everything. For example, many deterministic mathematical models of dynamic systems can exhibit chaotic behavior - which means that even the slightest error in measurement of the truth can lead to enormous deviations over time. This is why you can’t predict the weather. The world changes deterministically according to the law of physics, but similar starting conditions can lead to very different outcomes over time. This is what people refer to as the butterfly effect. A change as small as the flap of the wings of a butterfly might cause a hurricane in to future to happen, or not. If the butterfly is at the right place at the right time.

Getting back to the situation in which we need to provide a short explanation for something too complex, we have several options.

One option is to provide a metaphorical explanation, a shortcut, but make it clear that it isn’t what actually happens. This way they at least are aware that we aren’t giving them the full story. This can be the quickest approach, as long as our audience doesn’t keep on asking “Why?”

Another option is to ask our audience to bear with us while we develop the idea. This works well in college classes and other situations, but is not always available. Neither our audience, nor us have time for it, for everything. A benefit of this approach is that we’ll get to actually understand the concept better if we have to fully teach it.

Yet another option is is to find some middle ground in which we gradually build up the concept, and progressively close each leak in understanding. We can make the practical parts rock solid, and hand-wave around the non-important parts. It may be really hard to know which parts are important though.

There is no single right approach that works all the time. Read the situation.

Build vs. Buy -- Responding to All-or-Nothing Approaches

The Cranky Developer on Crater Moon Development — Mon, 16 Nov 2015 02:00:00 GMT

Technology, cost, and quality in deciding build vs buy

Continuous Integration, Misapplied

The Cranky Developer on Crater Moon Development — Sun, 15 Nov 2015 02:00:00 GMT

Putting together an continuous-integration-style automated build and deploy halfway through the project is exponentially more effort than starting off with one. At some point in the project, it may even be that the effort outweighs the gain.

LiveReload for Chrome extensions with Gulp

Sun, 15 Nov 2015 02:00:00 GMT

LiveReload is a fantastic feature to add to your development workflow to increase your productivity. The general idea is that when a change occurs to a file that makes up your app, your app is automatically rebuilt and the app restarts itself. This saves you from manually rebuilding and restarting your app each time you make a change and generally makes development much more pleasant. There are LiveReload implementations available for many languages and frameworks such as tiny-lr which can be used with Gulp for web development.

Matching arbitrary URL's to custom Firefox profiles

The Technical Blog of James on purpleidea.com — Sat, 14 Nov 2015 08:50:35 GMT

We’re constantly clicking on all sorts of different URL’s throughout the day. These clickable links appear in webpages (including in “web apps” like gmail) in mail clients like Evolution, in terminals such as GNOME-terminal, and any other GTK+ app on your GNU/Linux desktop. I wanted to perform custom actions when arbitrary URL’s are clicked, including running certain links in separate Firefox profiles. There are a bunch of different steps you have to do to get this working, but it should be easy to follow along. I’m doing all of this on Fedora 23, but it should work on other GNU/Linux environments.

HTTP/2 and SSL

infrequent oscillations — Fri, 13 Nov 2015 15:58:53 GMT

rjp.is is now HTTPS-only; and also supports HTTP/2 [FAQ] for future proofing. Here’s an owl to compensate for the entertainment-free nature of this post.

Links - November 13, 2015

Avy Faingezicht — Fri, 13 Nov 2015 02:00:00 GMT

2015-11-13

ho.dges.online — Fri, 13 Nov 2015 02:00:00 GMT

A&J rocking that Hollywood matinee idol look.

Fix Played Tracks with Empty Playcount in iTunes

Donat Studios — Thu, 12 Nov 2015 05:50:58 GMT

I have a quite large iTunes library, and I’m rather anal about keeping my metadata clean and up to date. In an earlier release of iTunes 12 there was a bug. Every so often after a song played, the Play Count would fail to increment, but the Last Played would update.

Searching my library, I found 74 tracks that had this issue.

I found this irritating and inaccurate, so I set forth finding a way to correct it.

I wrote the following simple AppleScript to set the playcount of such tracks to 1.

All you need to do is either paste the following into the 'Script Editor' or download it and run it. See video below if you need help running it once it’s open.

To view this video please enable JavaScript and upgrade to a web browser that supports HTML5 video

It’s secure… until it’s not

Dan Stroot — Thu, 12 Nov 2015 02:00:00 GMT

Is there a such thing as perfect security? In the 1770s Joseph Bramah created a lock that was vastly superior to any the world ever seen. Bramah believed it was 100% theft-proof. It was, and then it wasn't.

This post It’s secure… until it’s not first appeared on Dan Stroot's Blog

Das Keyboard Review

Anardil — Wed, 11 Nov 2015 22:30:00 GMT

Das Keyboard 4C Ultimate Review

Three years of (proprietary) projects

Separate Concerns — Wed, 11 Nov 2015 22:15:00 GMT

Some of the code I write is Open Source, but these days most of it is closed source and property of Moodstocks, the startup I work for. For the last three years I have had the chance to work on a lot of really interesting projects, the most significant of which I will talk about now. If you are still a student, maybe that can inspire you to join a startup. Maybe you will even decide to join us in our quest to advance mobile image recognition and applications in general.

Harvest, the Pikadeo crawler

When I joined Moodstocks in April 2010 the team was working on a mobile price comparison application called Pikadeo. The pitch was that you could take a picture of any cultural product (CD, DVD, book…) and it would give you a list of places where you could buy it, sorted by price.

The iOS application itself, largely designed by Louis Romero who had interned at Moodstocks and left just as I arrived, was working. The image recognition technology was working too, although it was purely server-side. The team was already researching how to leverage client-side processing but it was really just a crazy idea at that point, so Pikadeo was doing what most “mobile” image recognition software still does: send JPEG frames to the server.

What was missing was the data. We needed to crawl several large e-commerce websites, extract product images and metadata, send the former to the image recognition engine and the store the latter in a database. So I set out to write a crawler in Ruby, which was the dynamic language of choice of the team at the time. Moodstocks was a Ruby / C++ shop due to the background of the founders. Obviously things have changed a lot since then.

I tried to use Hadoop for the job, mostly because it was trending at that time and I had access to Amazon’s Elastic MapReduce. I soon understood that 1) the Hadoop Streaming interface was not quite there yet so I would have to switch to Java and 2) the Map/Reduce paradigm was not the best for the job anyway.

After reading a few papers on crawling (I had to anyway, since that project would be the basis of my MSc thesis, but it actually helped a lot) I ended up writing a kind of Master/Worker system, with work queues in Beanstalkd and metadata storage in Amazon’s SimpleDB, which did the job. It did the job a little too well, actually, since it ended up DDoSing an e-commerce website for a few seconds during a performance test for my thesis. Fortunately I was monitoring it and hit the stop button…

After setting reasonable speed limits and balancing the requests between various websites, Harvest was fast enough for our needs. The bottleneck became the image search engine itself, I will expand on that later.

Due to the deprecation of the Pikadeo product, Harvest is no longer used today. It is probably a good thing: it was my first Ruby program so the code was awful, it was way too complex and too tied to AWS (the master instance would run and kill worker instances, it relied a lot on SimpleDB…). That being said, the crawling model was sound.

Acorn

Once we got crawling sorted out, the image recognition engine itself became the problem. Oak (it has become a tradition to use plant-related names for our projects internally) had been almost entirely written by Cédric, Moodstocks’ CTO. It was a piece of C++ software, with the image recognition parts isolated in dynamic libraries and a Thrift layer to interface with the core Ruby on Rails Web application. It was multithreaded, designed to run on a single multicore EC2 instance.

The scalability pain point, it turned out, was not CPU load. The index was stored in a B+ Tree in Tokyo Cabinet, a tool we like a lot and still use today in other parts of our system. The problem was that when we indexed millions of images the dataset would inevitably become very large, larger than the available memory. The system would still be very responsive on most reads, but writes would invalidate large chunks of the in-memory cache and result in long pauses.

Latency is the enemy when you write image recognition software, so we decided not to sacrifice it: all the index had to fit in RAM. We decided to consider RAM as our primary datastore. That decision would bring about our later choices.

Since rebuilding an index can be very long we wanted something that could persist even if the engine crashed or had to be restarted for an update. Soon it became obvious that Redis could be the answer. However it was missing some commands that we needed, especially one that would insert the same key with different values in different maps (if you have already written inverted indices you may understand why, otherwise have a look at that presentation). Lua scripting was what we needed, but it wasn’t there yet so I ended up forking Redis to develop it in C while lobbying for scripting support. Acorn was, to my knowledge, the first application to run Redis 2.5 in production, and the first one to use Redis scripting too. We never encountered any Redis-related crash.

Now, to have the index fit entirely in RAM, we would have to distribute it across different machines, so Acorn would have to be a distributed system. Knowing that, I chose to make Acorn nodes single-threaded: they would communicate by message passing and we would have several of them per instance.

We chose MessagePack for serialization of Redis values, and I started looking at MessagePack-RPC. It had a lot of the pieces that I wanted for the distribution part, and one major problem: it was only usable in Ruby. But we were not CPU-bound… Would it be sensible to write the engine in a dynamic language? I started investigating that possibility. Our C++ libraries already had Ruby bindings that we used for vision R&D, and the little number crunching Oak Core did (mostly different scoring algorithms) turned out to be fast enough in Ruby.

So Acorn ended up as a distributed system written in Ruby, with MessagePack-RPC for communication and a fork of the development branch of Redis at its heart. It used MessagePack-RPC for communication with our Rails stack too.

In retrospect, relying on two unstable pieces of software was risky. It turned out well, and Redis was definitely the right choice, especially since Lua Scripting now allows us to use a regular, stable 2.6 version. 0MQ would probably have been a better technical choice than MessagePack-RPC, and plain C better than Ruby, but I believe those choices saved us development time, and time to market was important. Acorn is still in production today, doing its job for legacy clients who use online recognition.

Moodstocks Notes

You now know that when I joined Moodstocks it was trying to be a B2C company. However we were seeing interest in licensing our technology, and began to envision a B2B product: Moodstocks API.

When you write an API, especially as a product, you must write applications for it simultaneously. They serve two purposes: demonstrate what your API can do, and help you figure out how it should be improved. We set out to do that with two mobile applications, one of which was Notes.

The original idea I proposed was, I think, simple: Google SideWiki (RIP) for the real world. That is: you walk in the street, you see something interesting, you take a picture of it, you get a comments thread. If you are the first to do so, you get to leave the first comment (yay, first!!1).

As we were looking to add virality to it, that idea developed into a kind of mostly mobile social network where both people and objects could be followed. Objects actually had their own timeline with an associated Atom feed, which you could reach by browsing or, of course, as the result of an image search.

Technically the server-side part of Notes was a rather classic Sinatra application. The most interesting part of it was that it used some kind of CQRS architecture with all reads coming from Redis and all writes going to log-structured storage. The very nice thing about it was that any part of it could be replayed so it was almost trivial to reproduce bugs or replicate production incrementally to a development setup.

The iPhone application, on the other hand, was one of the nicest and most complex ones we have ever written. I wasn’t responsible for it so I won’t get into details here but the latest internal version we never actually released was IMO a thing of beauty.

As it turned out, Notes got a reasonable amount of online press after our CEO showed it to Michael Arrington at the Le Web conference. This got us a few users and we briefly thought about making it a product in its own right. I wrote a wxPython GUI to analyze logs, trying my hand for the first time at techniques like cohort analysis.

Eventually we took the decision not to invest more time in the idea: we were a small team and our now core B2B business needed our attention. Notes’ success was a long shot and would have required significant time and money investment so I guess it was the right decision, although I would love to see someone revisit the idea.

Moodstocks API v2

As I said, Notes was written to help us design our API. Using its feedback and that from the few users of our v1 API, which was more some kind of beta, I set out to write version two.

I will not expand too much on all its aspects here, REST-ish API design being well covered in the literature and online (start here).

The main differences with API v1 were the use of JSON instead of XML, and the ability to index a single image by uploading it to the API using multipart post. Previously, users would upload a XML list of image URLs and associated IDs; we would download them and tell you when indexing was over. Now users index single images and changes are taken into account instantly. The necessity for that was a lesson from Notes and user feedback, and it was made possible by Acorn.

Another interesting choice was the authentication method, HTTP Digest, which we kept from version one. Theoretically, it had all the right properties and was a standard, so it was the best choice. What we had not realized is how many implementations were broken or incomplete (i.e. not supporting nonce reuse, which is a necessity on mobile to reduce the number of HTTP requests). I ended up having to submit patches to a lot of them, and I am not even mentioning .NET land… If I had to do it again today I would probably go with Basic Auth and SSL.

Acorn Quantizer

Earlier, I wrote about how I had made Acorn processes single-threaded. This had some advantages, but also a big inconvenient.

Part of the image search process involves quantizing features, which means associating vectors in a many-dimensional space to integers. To do this the curse of dimensionality forces you to use an approximate nearest-neighbor search algorithm.

The way it works is: take a large number of features from a representative dataset and use some kind of clustering algorithm (e.g. k-means) on them to obtain a bunch of centroids (a “vocabulary”), then process these centroids to obtain a datastructure called kd-forest which will be used to perform nearest-neighbor search (a “dictionary”).

Vocabulary generation is clearly an offline task that requires a lot of number crunching and is done as little as possible. Generating the kd-forest, on the other hand, takes from a handful of seconds to a few minutes depending on the size of the vocabulary, so it is frequently done on engine startup. The kd-tree itself only exists in RAM.

The problem with that was that a kd-forest is a rather large datastructure. In our case it occupied hundreds of MB of RAM and took about one minute to generate. That was OK with Oak, where it was shared between threads, but with Acorn that overhead had to be paid for every process, both in space and time. We had to find a way to share the kd-forest across Acorn nodes on the same machine, and if possible to make startup faster.

The solution I opted for was to rewrite the whole quantizer. Previously we had been using popular Open Source libraries for this, but they didn’t do what I wanted.

I wrote the kd-forest generation algorithm as a LuaJIT program. It was the first Lua program officially used in production at Moodstocks, although as you will see it was only run offline. What it does is take centroids as input, generate a kd-forest and serialize it in a way easily readable in C thanks to the FFI. It can also actually perform nearest-neighbor searches but this is only used for test purpose.

Once the kd-forest is serialized, it can be loaded into system shared memory quite fast. A C library can then be used in every Acorn process to access this shared memory read-only and perform nearest-neighbor searches.

The idea is simple once you stop under-estimating the capabilities of SHM on Linux. By default it usually limited to a few MB so you have to increase it a lot for this to work (it can be done with sysctl). The implementation, on the other hand, is far from trivial. My code uses a lot of pointer arithmetics, I should probably clean it up someday, but in the meantime it does its job perfectly.

Seed

The Acorn Quantizer was the last major improvement to our online search stack. Around that time, we resolved on a major technological shift: we would perform image recognition on mobile devices directly instead of doing it on the server. Of course, initially, we would have an hybrid approach where on-device recognition would work as a kind of cache, but the mobile was where we would focus our efforts.

Doing on-device image recognition, though, almost meant starting from scratch: we had to make different technological trade-offs, use very different algorithms, and that meant writing an almost entirely new image recognition stack. We named that project Seed.

Seed encompasses a lot of things now, but at its core are proprietary Computer Vision algorithms that we set out to develop with Cédric and Maxime, who had joined us by then. We would discuss them as a team, then Maxime and Cédric would implement them in C while I would work on a Lua version.

The big picture is that some processing is done on the server at indexing time to generate signatures which are then sent to the client. Server-side software used to be entirely Lua, client-side software entirely C, but we decided to implement the whole stack in both languages. I think that was one of the best ideas we ever had. Being able to compare results avoided errors on both sides (tricky things like off-by-ones were always noticed thanks to the fact that Lua is 1-based, floating-point math issues were found…). Lua allowed faster prototyping on some parts and it was interesting to compare the different architectural choices we were making.

With the current (second) generation of the Seed algorithms, we are actually mostly using the C implementation through the LuaJIT FFI on the server side now. That is because I have been working on other projects while the rest of the team (which is not as comfortable with Lua) was developing them, so I would have been a bottleneck if we had kept the dual stack approach. I may well bring the Lua branch up to date someday though, who knows?

chksrv and chkcoherence

Moodstocks’ server-side architecture is some kind of SOA. That means we have a lot of different services that run as daemons and need to stay up. chksrv is a medium-sized program in Bash that takes care of this. It is deployed on every instance with a configuration file that indicates which services should be running on that instance, and it makes sure that they are (correctly). It also checks if other instances are up. If something goes wrongs, it warns the “ops team”, who is basically me and Cédric as a backup in case I am not available.

chksrv is a very useful piece of software but I was a bit worried by its growth as we added services. Standardizing the way we deamonize processes helped a lot with that by increasing code reuse (thank you libslack).

chkcoherence is the ideal complement to chksrv: where the latter checks if services are running, the former verifies that they are doing things right. It is also written in Bash at the top level. I have already written about its concept here.

Anemone

Anemone is the project that deals with everything related to metrics and measurements at Moodstocks. It is written in Lua and has quite a few different roles:

collect logs and data from production instances;
generate internal daily and weekly reports with business metrics for the platforms and email them to the team;
generate technical reports for operations (e.g. growth of different datasets);
generate weekly reports per application and email them to our platform customers;
generate custom reports tailored to the needs of our enterprise customers.

It also has a web-based dashboard for the team with high-level KPI, written in JavaScript and flot. Someday I might integrate Brett Slatkin’s Cohort Visualizer into it.

Dandelion

I said earlier that with Seed we generate image signatures on the server and send them to the mobile clients where they are used for recognition. Dandelion is the code name of the service responsible for that.

It turns out efficiently sending millions of image signatures per day, over slow and unreliable networks, to devices everywhere in the world, is not trivial. So Dandelion, more than software, is a synchronization protocol and its implementation; a range of tricks to make the best of mobile networks packaged as software. It is one of the reasons (along with all the innovation on CV algorithms and their optimized client-side implementation) why we can propose client-side recognition with databases of thousands of images or even videos, an order of magnitude more than our competitors.

The server part of Dandelion is written in Lua and depends on pieces like Redis and Beanstalkd, which is why I wrote haricot.

Physalis

Finally, Physalis is the project I am currently working on. It has not been released yet so I won’t get into the details, but I can explain the reasoning behind it.

While we were building Dandelion and through our experience with our clients, we learned the following things:

writing mobile applications that work offline but keep their data up to date is tricky;
most developers who attempt to implement it from scratch end up with a broken solution or just give up;
when you succeed it makes your applications a lot better.

So we thought: we have done it, why not make it accessible to everybody? This is what Physalis is: Moodstocks’ image signature distribution system generalized so that you can leverage it for your own mobile application.

Physalis will be available in private alpha for selected users soon, under its real brand name (Physalis is only its “internal plant-themed name”). If you are interested in trying it out, get in touch. The requirements are that you should be making a mobile application and ready to communicate on a regular basis with us: we are doing this alpha to collect useful feedback.

EDIT: Physalis was eventually released in August 2013 as Winch.

Strategy Design Pattern

denvaar's website — Wed, 11 Nov 2015 08:40:09 GMT

Quick example of the Strategy pattern in Python.

Troubleshooting Another Complex Performance Issue – Oracle direct path inserts and SEG$ contention

Tanel Poder Blog — Wed, 11 Nov 2015 02:35:23 GMT

Here’s an updated presentation I first delivered at Hotsos Symposium 2015.

It’s about lots of concurrent PX direct path insert ant CTAS statements that, when clashing with another bug/problem, caused various gc buffer busy waits and enq: TX – allocate ITL entry contention. This got amplified thanks to running this concurrent workload on 4 RAC nodes:

When reviewing these slides, I see there’s quite a lot that needs to be said in addition to what’s on slides, so this might just mean a (Powerpoint) hacking session some day!

Project Fi and Nexus 5X report

Pete's Points — Wed, 11 Nov 2015 02:30:08 GMT

So after having a Google Nexus 5 smartphone for the last 2+ years, when I saw that Project Fi had the potential to give me better coverage and a higher data limit for lower cost than T-Mobile, I decided to go for it.

Here's my notes on both the Project Fi service and the Google Nexus 5X smartphone.

Project Fi
- Value: great
  - cheap data rates and my modest usage means I have basically all the data, TXT, and voice I need for $40/month
- Coverage: better
  - Coming from T-Mobile I have the coverage I used to have plus access to Sprint and Wi-Fi calling so it's a no-brainer improvement on this front. Weirdly the heart of the downtown business district in my home town is a T-Mobile dead zone.
- Transition: flawless
- All the cell number porting stuff and account stuff went smooth as butter
- Overall it's really good service with no long-term contract, an unlocked phone, and a low monthly bill. Really pleased with this.
- There are some annoying limitations of Google Apps accounts vs regular gmail accounts. I use my main Google Apps accounts for most things, but they don't work with Google Play Store or Google Voice, so I need to use an old gmail account for that, which is annoying and confusing.
- Having voicemail just be there without having to dial a number and use an IVR touchpad is really nice. Amazing it took so long to reach this point.
Nexus 5X
- battery charges much faster and lasts much longer than Nexus 5. It will probably take me a while to adjust the chronic battery paranoia I learned using my Nexus 5 which could do ~6 hours at most.
- Camera is indeed better
- Fingerprint reader is nice
- Larger storage capacity is welcome
  - Still kind of mind-blowing that my Creative Nomad Xen mp3 player circa 2006 had double the storage of my current smartphone a decade later.
- Performance is not noticeable better compared to my Nexus 5. Most things are still a bit laggy when they shouldn't be, which the exception of the Calendar app which responds instantly and caches tons of data locally, which I wish every other app would do. It's disgusting that Google Inbox does not automatically sync my most recent 1GB of email at all times. I can use Here Maps and download a detailed map of the entire state of Colorado for offline use but getting the email with my flight details from last much while offline - not so much.
- The USB type C charger connection is a bit of a hassle as you are extremely likely to buy the wrong thing when shopping for chargers and cables on amazon at the moment.
  - Let Google Engineer Benson Leung's amazon reviews be your guide.
  - However, if you do indeed find the right thing, it's nice to see on the screen "Charging rapidly" as a confirmation.
  - I caught the dramatic price drop of the Nexus 5 after the initial batch last time around, but this time with the 5X I didn't wait. Less than a week after my order, google dropped the price significantly.
Android Marshmallow
- Overall a lot better
- Transition was a bit annoying as google does not transfer any app settings or OS settings other than your google accounts themselves. Had to go through app by app and adjust notifications and preferences
- Some fixes when connecting via bluetooth to my car stereo

So without too much time/effort configuring stuff, my smartphone is now in good working order and I have everything set up how I like and don't have to futz with it. Overall within the apps themselves I'm still feeling constantly frustrated by current UI trends of stacking stuff on top of each other, animations, and moving/disappearing tap targets.

Scars are beautiful

Steph Ango — Wed, 11 Nov 2015 02:00:00 GMT

I saw a wonderful exhibit of Ishiuchi Miyako’s photography. Her series “Scars” particularly struck me.

"Scars" by Ishiuchi Miyako

The images are hard to look at without wincing. The oversized prints were even more painful to witness in person. But after the initial shock washed over me, I began to appreciate the photography itself. There is beauty in these shapes.

Like a path winding through a forest, I get lost in the meandering edges of these scars, the delicate unevenness of each wrinkle. As I separate myself from the pain, there is an intriguing nature to these photographs that is undeniable. I find it impossible to look at these images without questioning the fetishism of perfection ingrained in us since youth.

I am fascinated by the Japanese philosophy wabi-sabi. It is a worldview centered around the beauty of transience. An example is kintsugi, the art of repairing broken pottery by joining the pieces together with gold lacquer. By doing so, the cracks are accentuated rather than covered up or discarded. The object becomes more valuable as it loses its “perfection”.

Example of a kintsugi cup

Cracks, like scars, tell a story. They are not only beautiful to look at, they are also lessons in survival and perseverance.

There is no learning, no victory, no good story without a share of failures, scars, and the perseverance it took to overcome them. Too often we overlook that idea in our perception of beauty.

Wabi-sabi does not have to be fatalistic. The appreciation of imperfection is not an invitation to let things break down and dilapidate. The opposite of fragility is not resilience. Some things become stronger or better when challenged by chaos and uncertainty.

Your muscles become stronger as they heal from being torn by physical exertion. Products and systems can be designed to not only withstand aging, but improve over time.

I find that much anxiety can be alleviated by finding beauty in transience, imperfection, and scars. If we feel judged for our scars, we become afraid to get scarred. We follow the path that results in the least amount of pain, rather than the path that leads to the most amount of learning.

Whether they are literal or figurative, do not fear the scars. Be proud to earn them. Gild them like a kintsugi tea cup.

Rebound: Season 2

🦉 olshansky 🦁 — Tue, 10 Nov 2015 15:54:42 GMT

Olshansky's review of Rebound: Season 2

Snaily Updates, Nov 10 - Family trees and inbreeding coefficient

Liza Shulyayeva — Tue, 10 Nov 2015 14:06:05 GMT

I did some research on snail ailments. This particular page was especially useful. Though SnailLife snails are not aquatic, some of these still seem very relevant: http://applesnail.net/content/various/snail_disease.php
Also did research on inbreeding in snails and complications it could cause. Apparently one commonly observed side effect is shell thinning. I then did some reading on finding the inbreeding coefficent and how to calculate it.
Looked into Google Visualization API to display the family trees and started a test implementation.
Started actually using Taiga.io, I’ve been wanting to try it out for a while.

Snake-duino: Hey look, a side project!

Joshua Rogers — Tue, 10 Nov 2015 14:00:00 GMT

My desk has a new ornament on it: an Arduino powered lamp. And why does that merit a blog post? Because I built it. After combining the Rainbowduino kit with some 600-ish points of solder, I found myself ruler of a 16MHz processor, 30Kb of instruction storage, 2Kb of data storage, and 64 tricolor pixels. And what does one do with 64 pixels, no inputs, and processing power reminiscent of a vintage entertainment console?

Template Method Design Pattern

denvaar's website — Tue, 10 Nov 2015 08:40:09 GMT

Example of the Template Method design pattern with Python.

Links - November 10, 2015

Avy Faingezicht — Tue, 10 Nov 2015 02:00:00 GMT

Listening to satellites for 30 dollars

Fabrice Aneche — Tue, 10 Nov 2015 00:51:43 GMT

I’ve ever dreamed of space & satellites, it turms out you can received pictures from them. After getting my radio amateur license in the US, I’ve discovered there was some satellites dedicated to radio amateurs but also some weather satellites from the late 90s still working and capable of sending pictures from the space like this one NOOA-15. SDR You don’t need expensive hardware anymore thanks to the SDR movement Software defined radio and some great developers, a simple 20$ USB key and some pratices are good enough to make it work.

Git aliases

Thought Eddies — Mon, 09 Nov 2015 19:31:00 GMT

Git aliases

Snaily Updates - Bug fixing and cleaning out old stuff

Liza Shulyayeva — Mon, 09 Nov 2015 14:39:54 GMT

Snaily Updates are going to be really brief posts with some short “daily” updates on SnailLife. I haven’t updated in a while because I’ve mostly just been refactoring and bug fixing. But it’s important, and even though I’m not working on new features right now I still want to document some of my progress and what I do each day.

PySpark dependencies

Thought Eddies — Mon, 09 Nov 2015 09:19:00 GMT

PySpark dependencies

Racket

Anardil — Mon, 09 Nov 2015 04:25:00 GMT

Thoughts on the Racket language

Links - November 09, 2015

Avy Faingezicht — Mon, 09 Nov 2015 02:00:00 GMT

How to cluster and failover (almost) anything - An intro to Pacemaker and Corosync

smcleod.net — Mon, 09 Nov 2015 02:00:00 GMT

Slides

Failover Demo

Chromebooks

Anardil — Sun, 08 Nov 2015 10:00:00 GMT

Review of Toshiba Chromebook 2

The Society of Mind & The Emotion Machine

june.kim — Sat, 07 Nov 2015 02:00:00 GMT

Electronic Voting

Separate Concerns — Sat, 07 Nov 2015 00:20:00 GMT

French politician Axelle Lemaire tweeted today that, like half of French voters, she was in favor of electronic voting. This tweet has resulted in a strong reaction among most of my techno-saavy friends which could be summed up as “OMG no, e-voting cannot be secure, please educate yourself”.

On this topic, my views differ with the majority opinion among people in technology, even though I support the same organizations fighting abuses in e-voting such as Electronic Frontier Foundation.

If you speak French, please read these three articles by LORIA reasearcher Véronique Cortier. They may change your mind on e-voting somehow. In particular, she argues that, although e-voting cannot be as secure as physical voting, it could very well be better than absentee voting.

Véronique is one of the researchers working on Belenios, which is Open Source and scientifically proven e-voting software. Belenios provides privacy, verifiability and full correctness, properties which many erroneously think cannot be satisfied simultaneously.

I am not denying that there are issues that will be hard or impossible to solve. The two main ones, for me, are the security of client terminals and the capacity for random citizen to trust the process. Indeed, understanding and verifying something like Belenios requires relative proficiency in math and computer science. I am willing to let the scientific community do that work, but some consider this view elitist.

However, I would like to see more research and experiments with e-voting. I think that making the organization of votes easier and cheaper will allow our political systems to evolve towards something where citizen have more direct influence on the democratic process.

I have to disclose that I know one of the authors of Belenios (Stéphane Glondu) personally, which is how I came to learn about systems like Helios and Belenios in the first place.

Some leaky abstractions in C++11

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Fri, 06 Nov 2015 21:14:04 GMT

Thanks to the introductory course in computer science taught by my colleague Filip Sadlo, I am starting to reacquaint myself more with the thought processes of beginners who learn C++ for the first time. Programming is all about abstraction. Finding out how to solve problems generally and generically instead of providing a bespoke solution for a single problem. This is certainly the intention of the Standard Template Library. As students, we all learned to use std::sort, std::transform, std::for_each, and so on. With the arrival of C++11, algorithms based on functors or predicates became even more useful. Instead of having to specify a function object beforehand, I can simply use an in-line lambda expression as the input of, say, std::remove_copy_if.

That’s all very well, but unfortunately, the STL and C++11 is somewhat “leaky”at times. Joel Spolsky coined the term ldquo;leaky abstraction” to refer to abstractions that, often inadvertently, break down and expose knowledge about the implementation to the user. During my—for want of a better word—“career”, I encountered multiple such leaks in the STL and C++. I want to discuss two interesting ones in detail here, because they tend to confuse beginners the most.

How to sort a list

Having seen the horrors of qsort() in the C library, you are glad that C++ offers std::sort. The following works great:

std::array<double, 3> A = { { 3.0, 2.0, 1.0 } };
std::sort( A.begin(), A.end() );

This works likewise:

std::vector<double> B = { 3.0, 2.0, 1.0 };
std::sort( B.begin(), B.end() );

You start feeling confident. Obviously, you have figured that one out. So:

std::list<double> C = { 3.0, 2.0, 1.0 };
std::sort( C.begin(), C.end() );

But the compiler denies you this small victory. Instead of the compiled code, you get pages of error messages. The salient points seems to be:

In file included from /usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/algorithm:62:
/usr/bin/../lib/gcc/x86_64-linux-gnu/4.9/../../../../include/c++/4.9/bits/stl_algo.h:1882:18: error: invalid operands to binary expression ('std::_List_iterator<double>' and
      'std::_List_iterator<double>')
      if (__last - __first > int(_S_threshold))

This is the way of the STL to tell you that the iterators of a list do not support subtraction. They are not random access iterators. The solution is to use the built-in sorting function for lists:

std::list<double> C = { 3.0, 2.0, 1.0 };
C.sort();

This feels cheap, does it not? Vector, for example, does not have this built-in function. Hence, you need to check the container type before applying a sort function to it. But this is completely contrary to the intention of the STL, which wants you to prefer generic algorithms over specific implementations.

Of course, this behaviour is well-documented and can easily be justified. It’s confusing for the beginner, nonetheless.

Copying functions

C++11 added the great std::function class, which permits us programmers to write less verbose code. For example, I often use a construction along the following lines:

std::function< double ( double ) > square = [] ( double a ) { return a*a; };
std::function< double ( double ) > root   = [] ( double a ) { return std::sqrt(a); };

// ... 

std::function< double ( double ) > f = square;
f( 2.0 );
f( 4.0 );

// ...

f = root;
f( 2.0 );
f( 4.0 );

The idea is to be able to reassign f later on in order to change the behaviour without writing new code. So far, so good. But how could we make this less verbose? By using auto, of course. This lets the compiler figure out the proper function signature:

auto square = [] ( double a ) { return a*a; };
auto root   = [] ( double a ) { return std::sqrt(a); };

// ... 

auto f = square;
f( 2.0 );
f( 4.0 );

// ...

f = root;
f( 2.0 );
f( 4.0 );

But, woe is us! Another compiler error:

f.cc:14:5: error: no viable overloaded '='
  f = root;

What happened? Well, we were being sneaky. The syntax we used actually refers to a lambda expression. In order to be able to create more compact and efficient code, whenever we specify it with auto, the type of the expression will be a special compiler-generated type. std::function, on the other hand, is more heavy-weight—it will typically incur some small overhead cost for calling. So, again, the compiler behaviour is justified (as always), but it certainly makes for some confusion of the sort “Why am I suddenly losing the ability to reassign function objects when using auto?”.

These are just two examples that I recently encountered. I am sure there are tons and tons more. I am not sure what would be better, from a pedagogical point of view:

Teaching these “leaks” along with the regular material
Waiting for students to encounter them on their own and be confused.

So far, I am using the second method. I try to justify this by the assumption that students learn more that way…

Internal Clojure libraries

Honza Pokorný — Fri, 06 Nov 2015 12:32:00 GMT

At work, we have a few Clojure services in production. Each service is its own leiningen project with its own dependencies. And because setting up a local maven repository is hard, each project reimplements quite a bit of logic. This duplicate code usually relates to the non-essential but still important parts of a service: logging, metrics, sentry integration, etc.

Just the other day, I was getting really annoyed with this situation, and was about ready to go learn about the wonderful intricacies of Maven, when I discovered leiningen’s install command.

With lein install, you can install your library as a jar and a pom to the local repository. Here, local repository means a local repository, typically in ~/.m2. Your apps can then depend on this library via the normal :dependencies list in the project file. This is all completely seamless and works well.

Apparently, this command has been around for a long time.

Armed with this new information, I was able to create an internal project called metrics and remove a ton of duplication. lein install also allows multiple versions of the same library to be installed at once. Simply require whatever version you need in your apps’ project.clj file.

This makes developing Clojure projects without having to publish your libraries to Clojars a lot easier.

Limitations of My Own Thinking

Josh Thompson — Thu, 05 Nov 2015 02:00:00 GMT

I sometimes make recommendations, or at least recount a story that has “actionable insights”. Anytime this happens, I start tripping over myself with warnings and qualifying statements.

Here’s what would happen:

I would make a recommendation (“start a side project to help get a better job”).
I would immediately follow it with a paragraph of qualifying statements. (“It’s not the side project itself that gets you the job, it’s the skills that you get while pursuing it. Well, not even the skills, but the “stick-to-itiveness” that you gain. But even if you’re naturally tenacious, you have to telegraph that to a potential employer, so really the side project is a way of showing traits that you already have. It helps you gain traits you might not have too. And helps you showcase both.” and on and on and on.)

That’s a rough way to write, and makes it hard to get a coherent thought down. So, now, whenever I make an assertion and want to qualify my statements, I’ll just link here. Every time I link to this page, hear my voice in your head saying:

“Please be aware that I’m only human and have limited knowledge. Important differences are invisible to both of us. These differences may cause your experience to differ wildly from mine”

On the limitation of my knowledge

We’re all suspicious of people who say but do not do. We’re also eager to hear from people that do, so we can learn from them and improve our own situation.

We should be wary of people who have done great things and then talk about it. We (humans) are unable to recall events and discern cause-and-effect with skill or accuracy.

When imparting advice, the advice giver andthe audience face three dangers:

Modify the advice or story to make yourself look better. This modification usually is not intentional.
Mistaking the relationship between what happened and what was observed. Have you noticed that “plain facts” get interpreted in opposite ways?
Misattribute cause and effect. “I started calling customers and changing the marketing to use the words and phrases they used, and our company grew 20% month over month!” Possible, but keep in mind that the number of people who drowned by falling into pools correlates with the number of films featuring Nicolas Cage.

Even with the best of intentions, it is impossible to view oneself and the events that ones has experienced with perfect clarity. I like to think well of myself, so with enough time (or wine) I’ll revise past events to make myself look better. This revision is to better protect my ego, and is insideous. I do this, and chances are good that you do too.

The change is subtle. Sometimes I’ll make a chance good thing seem a little more like something I chose. Or I’ll “misremember” my reaction to some bad thing as less immature and selfish than it really was.

This “misremembering” isn’t the end of the world. It just taints my ability to make generalizations and recommendations to other people. I’m not going to stop making recommendations. I am just leaving it up to you to be appropriately apprehensive and suspicious of everything that I write.

Conclusion

Don’t trust most of what most people say.

If you want a good starting point for trusting some people and distrusting others, I cannot recommend Antifragile highly enough. Go read it.

Links - November 05, 2015

Avy Faingezicht — Thu, 05 Nov 2015 02:00:00 GMT

Recover from an unresponsive Emacs

xenodium.com @alvaro — Wed, 04 Nov 2015 02:00:00 GMT

Wilfred Hughes has a handy tip to bail you out of a hung Emacs instance:

pkill -SIGUSR2 emacs

ps. Not had a chance to try it, but next time it happens…

Links - November 04, 2015

Avy Faingezicht — Wed, 04 Nov 2015 02:00:00 GMT

The Pink Startup

Sam Solomon — Wed, 04 Nov 2015 02:00:00 GMT

Since I started working on our fantasy sports product, nothing has been both praised and contested as much as our brand color.

Default to empathy

Steph Ango — Wed, 04 Nov 2015 02:00:00 GMT

There’s a saying you may have heard called Hanlon’s razor:

“Never attribute to malice what can be explained by stupidity.”

The word “malice” is perfect because it says nothing about the severity of the act. It could describe anything from someone cutting you off in traffic to an accident that blows up the Earth.

On the other hand, I find that “stupidity” only describes a narrow band of human behavior. The saying works just as well if you replace that word. For example:

Never attribute to malice what can be explained by…

a rough day at the office
a family emergency
an ill-fitting pair of pants
not having coffee this morning
a recent uptick in birds pooping on that person’s car

You get the idea.

Everyone has something on their mind. Something that isn’t going perfectly well in their life. A nagging issue that, however small, may affect your mood on any given day. We’ve all been that person who rudely cuts into traffic because it’s-been-a-long-stressful-day-and-if-we-could-just-be-home-right-now-we-could-finally-relax. And so yes, we will cut into traffic, because today we tell ourselves that we deserve to.

My secret weapon is my alter ego (well, one of my many alter egos, but we’ll get into that another day). I call him benefit-of-the-doubtman.

When someone does something seemingly malicious to me, I try to summon benefit-of-the-doubtman. Sometimes it’s hard to remember because my ego tends to bristle and roar: “Why is this happening to me?”

But benefit-of-the-doubtman comes to the rescue and asks: what are the conditions that caused this person to behave that way? Usually it’s easy to come up with one or two maybes. Maybe they’re worried about losing their job. Maybe they’re upset about something happening in their personal life.

There’s no room to take it personally. Instead of becoming defensive, I try to either disregard the apparent malice or resolve the causes behind it.

I often catch myself marveling at the concept of streets — cars whizzing by each other, divided only by a thin strip of paint. Streets say a lot about the power of ego. What makes this crazy system work is that none of us want to die. At least not in a head-on collision. The inherent sense of self-preservation and self-interest that we all share makes society possible, but also causes a lot of what seems like malice.

To solve the root cause of “malice”, it’s helpful to consider whether your environment is effective at converting self-interest into positive impact for the group. Maybe that environment is something you have the power to change?

Whether it’s true or not, I want to believe that most people are good people. I want to live in a world where we can all make that assumption. Call me an optimist, but I think that starts with defaulting to empathy.

What Programming And Computing Represent

Venam's Blog — Patrick Louis (Lebanon) — Wed, 04 Nov 2015 00:00:00 GMT

Hello fellow readers, In this post I'll discuss what programming and computing are for me. Computers are tools. Their functions can be summed to entertainment, utility, and information. It's common to find persons loosing themselves to a tool, slave to their creation. Programming is something useful. It's a big domain where you learn languages to instruct a machine to take actions. Those instructions can range from outputting things (maybe screen pixels but not limited to that), taking inputs from the outside world (not necessarily a keyboard or mouse), doing calculations at very high speed, storing data, and transferring data.

Calling Matlab from Python

bastibe.de — Tue, 03 Nov 2015 17:05:00 GMT

For my latest experiments, I needed to run both Python functions and Matlab functions as part of the same program. As I noted earlier, Matlab includes the Matlab Engine for Python (MEfP), which can call Matlab functions from Python. Before I knew about this, I created Transplant, which does the very same thing. So, how do they compare?

Usage

As it's name suggests, Matlab is a matrix laboratory, and matrices are the most important data type in Matlab. Since matrices don't exist in plain Python, the MEfP implements it's own as matlab.double et al., and you have to convert any data you want to pass to Matlab into one of those. In contrast, Transplant recognizes the fact that Python does in fact know a really good matrix engine called Numpy, and just uses that instead.

       Matlab Engine for Python        |              Transplant
---------------------------------------|---------------------------------------
import numpy                           | import numpy
import matlab                          | import transplant
import matlab.engine                   |
                                       |
eng = matlab.engine.start_matlab()     | eng = transplant.Matlab()
numpy_data = numpy.random.randn(100)   | numpy_data = numpy.random.randn(100)
list_data = numpy_data.tolist()        |
matlab_data = matlab.double(list_data) |
data_sum = eng.sum(matlab_data)        | data_sum = eng.sum(numpy_data)

Aside from this difference, both libraries work almost identical. Even the handling of the number of output arguments is (accidentally) almost the same:

       Matlab Engine for Python        |              Transplant
---------------------------------------|---------------------------------------
eng.max(matlab_data)                   | eng.max(numpy_data)
>>> 4.533                              | >>> [4.533 537635]
eng.max(matlab_data, nargout=1)        | eng.max(numpy_data, nargout=1)
>>> 4.533                              | >>> 4.533
eng.max(matlab_data, nargout=2)        | eng.max(numpy_data, nargout=2)
>>> (4.533, 537635.0)                  | >>> [4.533 537635]

Similarly, both libraries can interact with Matlab objects in Python, although the MEfP can't access object properties:

       Matlab Engine for Python        |              Transplant
---------------------------------------|---------------------------------------
f = eng.figure()                       | f = eng.figure()
eng.get(f, 'Position')                 | eng.get(f, 'Position')
>>> matlab.double([[ ... ]])           | >>> array([[ ... ]])
f.Position                             | f.Position
>>> AttributeError                     | >>> array([[ ... ]])

There are a few small differences, though:

Function documentation in the MEfP is only available as eng.help('funcname'). Transplant will populate a function's __doc__, and thus documentation tools like IPython's ? operator just work.
Transplant converts empty matrices to None, whereas the MEfP represents them as matlab.double([]).
Transplant represents dict as containers.Map, while the MEfP uses struct (the former is more correct, the latter arguable more useful).
If the MEfP does not know nargout, it assumes nargout=1. Transplant uses nargout(func) or returns whatever the function writes into ans.
The MEfP can't return non-scalar structs, such as the return value of whos. Transplant can do this.
The MEfP can't return anonymous functions, such as eng.eval('@(x, y) x>y'). Transplant can do this.

Performance

The time to start a Matlab instance is shorter in MEfP (3.8 s) than in Transplant (6.1 s). But since you're doing this relatively seldomly, the difference typically doesn't matter too much.

More interesting is the time it takes to call a Matlab function from Python. Have a look:

This is running sum(randn(n,1)) from Transplant, the MEfP, and in Matlab itself. As you can see, the MEfP is a constant factor of about 1000 slower than Matlab. Transplant is a constant factor of about 100 slower than Matlab, but always takes at least 0.05 s.

There is a gap of about a factor of 10 between Transplant and the MEfP. In practice, this gap is highly significant! In my particular use case, I have a function that takes about one second of computation time for an audio signal of ten seconds (half a million values). When I call this function with Transplant, it takes about 1.3 seconds. With MEfP, it takes 4.5 seconds.

Transplant spends its time serializing the arguments to JSON, sending that JSON over ZeroMQ to Matlab, and parsing the JSON there. Well, to be honest, only the parsing part takes any significant time, overall. While it might seem onerous to serialize everything to JSON, this architecture allows Transplant to run over a network connection.

It is a bit baffling to me that MEfP manages to be slower than that, despite being written in C. Looking at the number of function calls in the profiler, the MEfP calls 25 functions (!) on each value (!!) of the input data. This is a shockingly inefficient way of doing things.

TL;DR

It used to be very difficult to work in a mixed-language environment, particularly with one of those languages being Matlab. Nowadays, this has thankfully gotten much easier. Even Mathworks themselves have stepped up their game, and can interact with Python, C, Java, and FORTRAN. But their interface to Python does leave something to be desired, and there are better alternatives available.

If you want to try Transplant, just head over to Github and use it. If you find any bugs, feature requests, or improvements, please let me know in the Github issues.

List of Awesomeness #3

THINK@RJ — Tue, 03 Nov 2015 17:00:00 GMT

Shifting gears a bit, I have decided to add in a few papers,books, articles that I found quite riveting last few weeks.

Papers

Starting at the beginning – with the greatest minds of our century – A. Einstein. He published 5 papers in a single year, 1905 a.k.

Links - November 03, 2015

Avy Faingezicht — Tue, 03 Nov 2015 02:00:00 GMT

Prototype: A GUI-friendly Snappy

kyrofa's blog — Tue, 03 Nov 2015 02:00:00 GMT

So this is the week of the Ubuntu Online Summit, and many of the sessions are discussing Snappy. As you may know, Snappy is currently pretty geared toward embedded, headless devices. However, it is the successor to Click, and eventually the phones will be based upon it. To drive that effort forward, a few colleagues and I had a session (update: I used to link to the video here, but the Summit server is no more) where we discussed the path forward for supporting snaps on other devices, specifically the phone and the desktop.

Compiling QEMU with GlusterFS Support on Ubuntu 14.04

Mustafa Akın — Tue, 03 Nov 2015 02:00:00 GMT

On Ubuntu 14.04 (and maybe later) Qemu does not come with GlusterFS support. To compile it, you have to fetch the sources and enable flags. You also need to get the build dependencies, and some debian package tools. You can install the dependencies as follows: $ sudo apt-get install build-essential glusterfs-common devscripts dpkg-dev $ sudo apt-get build-dep qemu You also need to execute the following command, but it might be better to create a script file and execute it:

Explaining Go error handling

divan's blog — Mon, 02 Nov 2015 17:36:54 GMT

I recently translated great article — Errors are values by Rob Pike — and we discussed it in our podcast Golangshow (in russian). One thing I was surprised about is that even experienced Go developers sometimes do not understand the core idea of that article.

Looking back, I remember my first impressions when I read it for the first time. It was similar to “It looks like Pike just adds some complexity to what could’ve been solved gracefully with exceptions”. I have never been fond of exceptions, but that’s the first thought I remember. The example in the article was clearly asking for comparison with exceptions’ way to deal with errors and it didn’t look like a winner here.

Language choice

Honza Pokorný — Mon, 02 Nov 2015 17:25:00 GMT

Warning: this is a rant

Recently, I have made great progress in my journey towards Haskell enlightenment. I finally see how many of the little pieces of the Haskell puzzle fit together. At this point, I feel empowered to go forth and write useful programs. I read through the source of Scotty the web framework the other day, and I was very pleasantly surprised that I understood how it works. I absolutely love Haskell. I love that it makes you think. One does not simply open a text editor and start banging at the keyboard to write a Haskell program. I love that Haskell encourages generalizations and abstractions. One of the biggest heureka moments in my journey was understanding the full implications of why a function of type a -> a has a single implementation. I’m addicted to running my program for the first time (after fighting with the compiler for ages), and having it work. I think monad transformers and lenses are really clever. By many criteria, Haskell is the perfect programming language.

It has taken me four years to get here.

I used to get so discouraged that I took breaks for weeks or months at a time because I didn’t see the point of continuing. But I always came back. Now I have finally arrived. I would say I’m an intermediate Haskeller. Naturally, I’m thinking about writing some Haskell code at work which is going to be easy given our service-oriented architecture.

I have also been playing with Purescript which is a Haskell dialect that compiles to javascript. In many ways, Purescript is a much better Haskell because it doesn’t come with the historical baggage. In speaking with my colleague who doesn’t know Purescript about introducing it into our code base, I realized the gravity of what I was asking him to learn. It sounds great to say “let’s rewrite this in purescript” and expect someone to come back from their weekend having learned enough to be dangerous when it took me four years to learn.

Another great example is the open source community. If you choose Haskell for your open source project, you might be productive, safe to refactor, write little code — but how many people will be willing to learn Haskell to contribute a fix or a new feature?

Many of my Haskell friends like to mock the Go programming language. Myself included at times. Mind you, the language is objectively poorly designed. The error handling, the lack of generics, the ridiculous package manager, the absurd type system, the range thing, etc. It’s almost exactly the opposite of Haskell.

And yet, Go is a lot more popular than Haskell according to GitHub. Yet, there are so many amazing projects written in Go, like Docker, Influxdb, etcd, consul, prometheus, packer, and many more. Unlike Haskell, if you ask your coworkers to learn Go over the weekend, everyone will come back with a little app they built. A clearly inferior tool is used by crowds of people to build cool things.

What should we conclude from this? The choice of programming language matters. Programming is a social activity. Fewer features seems to equal easier to learn. Generalization and programming language innovation seem to be out of favor. Creating software to solve real problems with blunt tools seems to be a lot more important than using a sharp axe. We’d much rather use an inferior tool whose manual we don’t have to read. We’d much rather snap a picture with our smartphone than to learn how to use a DSLR.

Setting up an sbt-based project for Scala 2.11 with ScalaTest support

Home on The Log Book of Manuel Kießling — Mon, 02 Nov 2015 17:13:00 GMT

This post describes how to set up a new Scala 2.11.7 project with ScalaTest 2.2.4 support using sbt 0.13.9 on Mac OS X 10.11 “El Capitan” with Java 1.8. If set up as describes below, you can manage your project via sbt, and you will be able to run your test cases via sbt test. Also, after setting things up this way, the project can be opened and used in IntelliJ IDEA 14 as an sbt project.

Links - November 02, 2015

Avy Faingezicht — Mon, 02 Nov 2015 02:00:00 GMT

Margin of Safety: Risk-Averse Value Investing Strategies for the Thoughtful Investor

🦉 olshansky 🦁 — Mon, 02 Nov 2015 02:00:00 GMT

Olshansky's review of Margin of Safety: Risk-Averse Value Investing Strategies for the Thoughtful Investor by Seth A. Klarman

Amounts 2, In Review

Swift Fox Software LLC — Sun, 01 Nov 2015 15:28:07 GMT

After nearly two years in (Part time) development, Amounts 2 has finally been sent off to Apple for review. This will be a free update to all users, and I think everyone will love it. The UI has been overhauled to support the new screen sizes, icons have been updated, and now the app supports... Read more »

The post Amounts 2, In Review first appeared on Swift Fox Software LLC.

Sunlight

Anardil — Sun, 01 Nov 2015 09:00:00 GMT

Short poem

I couldn’t agree more.

🦉 olshansky 🦁 — Sun, 01 Nov 2015 03:47:57 GMT

Having grown up in a reform Jewish household, I always enjoyed the holidays and just having a reason to get together as a community and…

Tiny Keep (#blogtober 31)

infrequent oscillations — Sun, 01 Nov 2015 01:08:55 GMT

THE END. Tiny Keep has the distinction of being the last. But sadly not the best. This is actual a fair reflection of me this evening. They’ve bizarrely decided to use normal* console controls - left stick does movement, right stick controls the camera. This makes it nigh-on impossible to control reliably, especially when you get to combat later. Managed to set myself and a bunch of enemies on fire. Oops.

Inglourious Basterds

🦉 olshansky 🦁 — Sat, 31 Oct 2015 17:26:16 GMT

Olshansky's review of Inglourious Basterds

Ten days in Spain - Madrid & Seville

Jonathan Jones homepage: planetjones.net — Sat, 31 Oct 2015 02:00:00 GMT

With ten days to enjoy Spain we eventually succeeded in buying a Renfe Spain pass and made our way to Madrid and Seville.

No Brakes Valet (#blogtober 30)

infrequent oscillations — Sat, 31 Oct 2015 01:55:00 GMT

Despite the name, you do have brakes. I didn’t realise this for quite a few games… Easy enough - park the cars in the parking bays. Er, yeah. Don’t ram the explosives truck with another vehicle, btw. I cannot even.

Pixels

🦉 olshansky 🦁 — Fri, 30 Oct 2015 20:16:46 GMT

Olshansky's review of Pixels

My Livecoding.tv account deletion saga

Liza Shulyayeva — Fri, 30 Oct 2015 19:23:49 GMT

Update 3, Nov 2 2015 (PM): Well, it’s been…strange. First, it seems my account was finally deleted by Livecoding.tv (thank you!) After being up all night with libel posted all over it, hours after Jamie Green (the other Livecoding co-founder) and Sam Altman (Y Combinator president) replied to the Hacker News thread claiming they were trying to do something, something finally happened. I have heard nothing from either of them since their original comments and am not sure exactly what spurred the removal of slander from the profile and its eventual deletion. I know some other Livecoding users quit and had a discussion with the CEO in live chat also.

The Fountain

🦉 olshansky 🦁 — Fri, 30 Oct 2015 09:58:26 GMT

Olshansky's review of The Fountain

searchcode local

Ben E. C. Boyter — Fri, 30 Oct 2015 09:56:37 GMT

I am going to copy the searchcode pitch itself below quickly before explaining it a bit further.

“searchcode offers powerful code search over billions of lines of open source code. Imagine what it could do with your private repositories.

There have been requests to offer a downloadable version of searchcode. Given enough interest a downloadable hostable version of searchcode will be offered. Register your email below to register your interest.

Upgrade your job

Josh Thompson — Fri, 30 Oct 2015 02:00:00 GMT

So, apparently I send a lot of email about people trying to get cool jobs. Here’s yet another email I sent to a friend, recorded here.

Hi [redacted],

First I want to highlight is that flexible/remote jobs are just like normal jobs, but more people want them, so the companies can be highly selective in their hiring.

So, it is quite possible that your next job will be (and should be) a “normal” 9-5, where you’re in the same office as everyone else.

This company may not be particularly progressive or innovative (however you’d define those) but it could be a perfect playground to build your own personal portfolio of incredible work. You could spend a year at a company, do great things, and then take those things to another company that allows all the travel and flexible work.

A really good book ( So Good They Can’t Ignore You, I recommend reading this ASAP) explains this as “career capital”. You have to gain career capital to “turn it in” for a good job. Most people want a good job, but most people don’t have any career capital.

At your next job, whatever it is, you should relentlessly pursue this career capital. You’ll then be in a position to do exactly what you want to do next.

I forgot to tell you the most significant part of what I learned when trying to pick up a kick-ass job.

It all started clicking when I stumbled across a guy named Ramit Sethi.

Far and away, he’s the most influential person on me and the work that I do. Everything that I say that companies like I’ve basically lifted from him.

He talks about many things, but the relevant bits that I started with were “Natural Networking” and “Interviewing”

Here’s a long list of links. I’d set aside 30 mins and look over stuff, then dig deeper into what he has to say. Seriously. Everything is gold. Your “this guy is scammy” radar should be going off because of the URL and titles he uses, but read/watch.

Natural networking:

Interviewing:

If you could do only one thing, I’d dig deep into the “natural networking” side of stuff, and study how he uses/recommends “scripts” and having some go-to and practiced concise stories.

-Josh

Constraint solver

Matt Keeter — Fri, 30 Oct 2015 02:00:00 GMT

Gradient descent in Haskell

A JavaScript Modules Manager That Fits in a Tweet

Baptiste Fontaine’s Blog — Fri, 30 Oct 2015 00:58:00 GMT

ES6 does provide modules; but unless you’re using Babel you’ll have to rely on third-party libraries such as RequireJS until all major browsers support them.

I use D3 everyday to visualize data about ego networks and have a small (400-500 SLOC) JavaScript codebase I need to keep organized. In the context I work in I must keep things simple as I won’t always be there to maintain the code I’m writing today.

How simple a modules implementation could possibly be? It should at least be able to register modules and require a module inside another; much like Python’s import. It should also handle issues like circular dependencies (e.g. foo requires bar which requires foo) and undeclared modules. Modules should be lazily loaded, i.e. only when they are required; and requiring twice the same module shouldn’t execute it twice.

Well, here is one:

p={f:{},m:{},r:function(a,b){p.f[a]=b},g:function(a){if(!p.m[a]){if(p.m[a]<1|!p.f[a])throw"p:"+a;p.m[a]=0;p.m[a]=p.f[a](p)}return p.m[a]}};

It’s 136-bytes long. 139 if you count the variable definition. At this level you can’t expect long function names but here is an usage example:

// register a "main" module. A module consists of a name and a
// function that takes an object used to require other modules.
p.r("main", function(r) {
    // get the "num" module and store it in a `num` variable
    var num = r.g("num");

    // use it to print something
    console.log(num.add(20, 22));
});

// register a "num" module
p.r("num", function(r) {
    // a module can export bindings by returning an object
    return {
        add: function(a, b) { return a+b; },
    };
});

// call the "main" module
p.g("main");

This code will print 42 in the console. It only uses two modules but the implementation works with an arbitrary number of modules. A module can depend on any number of other modules that can be declared in an arbitrary order.

Consider this example:

p.r("m1", function(r) { r.g("m2"); });
p.r("m2", function(r) { r.g("m3"); });
p.r("m3", function(r) { r.g("m1"); });

p.g("m1");

m1 depends on m2 which depends on m3 which itself depends on m1. The implementation won’t die in an endless loop leading to a stack overflow but will fail as soon as it detects the loop:

p:m1

Admittedly this error message doesn’t give us too information but we have to be thrifty in order to fit under 140 characters. The prefix p: tells you the error comes from p, and the part after is the faulty module. It can either be a wrong name (the module doesn’t exist) or a circular dependency.

Walk-through

Note: don’t use this at home. This is just an experiment; I eventually used Browserify for my project.

We need an object to map modules to their functions; we’ll populate it on calls to register. We need another object to store the result of their function call; i.e. what they export. I added a third object to “lock” a module while it’s executed in order to detect circular dependencies.

We’ll have something like that:

var p = {
    _fn: {},   // the functions
    _m: {},    // the modules’ exported values
    _lock: {}, // the locks

    register: function(name, callback) {
        // add the function in the object
        p._fn[name] = callback;
    },

    get: function(name) {
        // if we have a value for this module let’s return it.
        // Note that we should use `.hasOwnProperty` here
        // because this’ll fail if the module returns a falsy
        // value. This is not really important for this problem.
        if (p._m[name]) {
            return p._m[name];
        }

        // if it’s locked that’s because we’re already getting
        // it; so there’s a recursive requirement
        if (p._lock[name]) {
            throw "Recursive requirement: '" + name + "'";
        }

        // if we don’t have any function for this we can’t
        // execute it and get its value. See also the
        // remark about `.hasOwnProperty` above.
        if (!p._fn[name]) {
            throw "Unknown module '" + name + "'";
        }

        // we lock the module so we can detect circular
        // requirements.
        p._lock[name] = true;

        try {
            // execute the module's function and pass
            // ourselves to it so it can require other
            // modules with p.get.
            p._m[name] = p._fn[name](p);
        } finally {
            // ensure we *always* remove the lock.
            delete p._lock[name];
        }

        // return the result
        return p._m[name];
    },
};

This works and is pretty short; but that won’t fit in a Tweet ;)

Let’s compact the exceptions into one because those strings take a lot of place:

if (p._lock[name] || !p._fn[name]) {
    throw "Module error: " + name;
}

The error is less explicit but we’ll accept that here.

We try to get as little code as possible then use YUI Compressor to remove the spaces and rename the variables. This means we can still work with (mostly) readable code and let YUI Compressor do the rest for us.

I measure the final code size with the following command:

yuicompressor p.js | wc -c

Right now we have 240 bytes. We need a way to remove 100 bytes. Let’s rename the attributes. _fn becomes f; _m becomes m, _lock becomes l and the public methods are reduced to their first letter. We can also remove the var since p will be global anyway. Let’s also reduce the error message prefix to "p:".

p = {
    f: {}, m: {}, l: {},

    r: function(name, callback) { p.f[name] = callback; },

    g: function(name) {
        if (p.m[name]) {
            return p.m[name];
        }

        if (p.l[name] || !p.f[name]) {
            throw "p:" + name;
        }

        p.l[name] = true;

        try {
            p.m[name] = p.f[name](p);
        } finally {
            delete p.l[name];
        }

        return p.m[name];
    },
};

That’s 186 bytes once compressed. Not bad! Note that we have twice the same line in the g function (previously known as “get”):

return p.m[name];

We can invert the first if condition and fit the whole code in it; combining both returns into one. This is equivalent to transforming this code:

function () {
    if (A) {
        return B;
    }

    // ...
    return B;
}

Into this one:

function () {
    if (!A) {
        // ...
    }

    return B;
}

The first form is preferable because it removes one indentation level for the function body. But here return is a keyword we can’t compress.

Speaking of keyword we can’t compress; how could we remove the delete? All we care about is to know if there’s a lock or not, so we can set the value to false instead, at the expense of more memory. This saves us only one byte but since we only care about the boolean values we can replace true with 1 and false with 0.

We’re now at 166 bytes and the g function looks like this:

function(name) {
    if (!p.m[name]) {
        if (p.l[name] || !p.f[name]) {
            throw "p:" + name;
        }

        p.l[name] = 1;

        try {
            p.m[name] = p.f[name](p);
        } finally {
            p.l[name] = 0;
        }
    }

    return p.m[name];
}

Now, what if we tried to remove one of the three objects we’re using? We need to keep the functions and the results in separate objects but we might be able to remove the locks object without losing the functionality.

Assuming that modules only return objects let’s merge m and l. We’ll set p.m[A] to 0 if it’s locked and will then override the lock with the result. p.m[A] then have the following possible values:

undefined: the key doesn’t exist; the module hasn’t been required yet
0: the module is currently being executed
something else: the module has already been executed; we have its return value

We need to modify our code a little bit for this:

function(name) {
    if (!p.m[name]) {
        if (p.m[name] === 0 || !p.f[name]) {
            throw "p:" + name;
        }

        p.m[name] = 0;
        p.m[name] = p.f[name](p);
    }

    return p.m[name];
}

Note that this allowed us to get ride of the try/finally which let us go down to 143 bytes. We can already save two bytes by using < 1 instead of === 0.

Replacing || (boolean OR) with | (binary OR) saves one more byte and allows us to fit in 140 bytes! We can go further and remove the brackets for the inner if since it only has one instruction. We need to do that after the compression because YUI Compressor adds brackets if they’re missing.

The final code looks like this:

p = {
    f: {}, m: {},

    r: function(name, callback) { p.f[name] = callback; },

    g: function(name) {
        if (!p.m[name]) {
            if (p.m[name] < 1 | !p.f[name])
                throw "p:" + name;

            p.m[name] = 0;
            p.m[name] = p.f[name](p);
        }

        return p.m[name];
    },
};

That’s 139 bytes once compressed! You can see the result at the top of this blog post.
Please add a comment below if you think of any way to reduce this further while preserving all existing features.

Thank you for reading!

Hatoful Boyfriend (#blogtober 29)

infrequent oscillations — Thu, 29 Oct 2015 22:44:47 GMT

I’m a girl. At a school for pigeons. My best friend is a rock dove. Here’s my teacher. He’s a quail. OBVIOUSLY. Apparently I live in a cave. This might be a side plot about pudding. Or it might be just another thing which is making my brain bleed out. Oh ho ho, REFERENCES. “RAGE VOLTAGE” is an excellent metaphor but everyone knows that it’s really Amps that do the damage.

SQL Monitoring in Oracle Database 12c

Tanel Poder Blog — Thu, 29 Oct 2015 20:53:08 GMT

Here’s my latest OOW presentation – SQL Monitoring in Oracle Database 12c:

[direct link]

You can download all my scripts from /files/

Massive Memory Leak in the Matlab Engine for Python

bastibe.de — Thu, 29 Oct 2015 14:18:00 GMT

As of Matlab 2014b, Matlab includes a Python module for calling Matlab code from Python. This is how you use it:

import numpy
import matlab
import matlab.engine

eng = matlab.engine.start_matlab()
random_data = numpy.random.randn(100)
# convert Numpy data to Matlab:
matlab_data = matlab.double(random_data.tolist())
data_sum = eng.sum(matlab_data)

You can call any Matlab function on eng, and you can access any Matlab workspace variable in eng.workspace. As you can see, the Matlab Engine is not Numpy-aware, and you have to convert all your Numpy data to Matlab double before you can call Matlab functions with it. Still, it works pretty well.

Recently, I ran a rather large experiment set, where I had a set of four functions, two in Matlab, two in Python, and called each of these functions a few thousand times with a bunch of different data to see how they performed.

While doing that I noticed that my Python processes were growing larger and larger, until they consumed all my memory and a sizeable chunk of my swap as well. I couldn't find any reason for this. None of my Python code cached anything, and the sum total of all global variables did not amount to anything substantial.

Enter Pympler, a memory analyzer for Python. Pympler is an amazing library for introspecting your program's memory. Among its many features, it can list the biggest objects in your running program:

from pympler import muppy, summary
summary.print_(summary.summarize(muppy.get_objects()))

                                      types |   # objects |   total size
=========================================== | =========== | ============
                        <class 'array.array |        1076 |      2.77 GB
                                <class 'str |       42839 |      7.65 MB
                               <class 'dict |        8604 |      5.43 MB
                      <class 'numpy.ndarray |          48 |      3.16 MB
                               <class 'code |       14113 |      1.94 MB
                               <class 'type |        1557 |      1.62 MB
                               <class 'list |        3158 |      1.38 MB
                                <class 'set |        1265 |    529.72 KB
                              <class 'tuple |        5129 |    336.98 KB
                              <class 'bytes |        2413 |    219.48 KB
                            <class 'weakref |        2654 |    207.34 KB
            <class 'collections.OrderedDict |          65 |    149.85 KB
                 <class 'wrapper_descriptor |        1676 |    130.94 KB
  <class 'traitlets.traitlets.MetaHasTraits |         107 |    123.55 KB
                  <class 'getset_descriptor |        1738 |    122.20 KB

Now that is interesting. Apparently, I was lugging around close to three gigabytes worth of bare-Python array.array. And these are clearly not Numpy arrays, since those would show up as numpy.ndarray. But I couldn't find any of these objects in my workspace.

So let's get a reference to one of these objects, and see who they belong to. This can also be done with Pympler, but I prefer the way objgraph does it:

import array
# get a list of all objects known to Python:
all_objects = muppy.get_objects()
# sort out only `array.array` instances:
all_arrays = [obj for obj in all_objects if isinstance(obj, array.array)]

import objgraph
objgraph.show_backrefs(all_arrays[0], filename='array.png')

It seems that the array.array object is part of a matlab.double instance which is not referenced from anywhere but all_objects. A memory leak.

After a bit of experimentation, I found the culprit. To illustrate, here's an example: The function leak passes some data to Matlab, and calculates a float. Since the variables are not used outside of leak, and the function does not return anything, all variables within the function should get deallocated when leak returns.

def leak():
    test_data = numpy.zeros(1024*1024)
    matlab_data = matlab.double(test_data.tolist())
    eng.sum(matlab_data)

Pympler has another great feature that can track allocations. The SummaryTracker will track and display any allocations between calls to print_diff(). This is very useful to see how much memory was used during the call to leak:

from pympler import tracker
tr = tracker.SummaryTracker()
tr.print_diff()
leak()
tr.print_diff()

                     types |   # objects |   total size
========================== | =========== | ============
       <class 'array.array |           1 |      8.00 MB
...

And there you have it. Note that this leak is not the Numpy array test_data and it is not the matlab array matlab_data. Both of these are garbage collected correctly. But the Matlab Engine for Python will leak any data you pass to a Matlab function.

This data is not referenced from anywhere within Python, and is counted as leaked by objgraph. In other words, the C code inside the Matlab Engine for Python copies all passed data into it's internal memory, but never frees that memory. Not even if you quit the Matlab Engine, or del all Python references to it. Your only option is to restart Python.

Postscriptum

I since posted a bug report on Mathworks, and received a patch that fixes the problem. Additionally, Mathworks said that the problem only occurs on Linux.

Eve 0.6.1 has been released

Nicola Iarocci — Thu, 29 Oct 2015 02:00:00 GMT

A new version of Eve, the REST API framework for Humans, has been released today. Following the 0.6 milestone released one month ago, v0.6.1 introduces some fixes and few new important features. Upgrade is strongly encouraged. As always, a changelog with full list of updates is available.

Links - October 29, 2015

Avy Faingezicht — Thu, 29 Oct 2015 02:00:00 GMT

Why Docker?

Brightball Articles — Thu, 29 Oct 2015 02:00:00 GMT

Containers are not a new thing, but implementing them was always a little more complicated than it needed to be. Docker made great leaps in simplification of containers and set the world on fire from there. Let’s look at why.

Pesky "AMD-Action:authenticate:SP" and its Fix

Graham Stevens – Grh.am — Thu, 29 Oct 2015 02:00:00 GMT

Over the last few weeks, I’ve been plagued with an issue within the OSX El Capitan App Store, that resulted in me being unable to install any updates or install/purchase new applications. At first, I wondered if this was due to my hasty upgrade to the GM of OSX 10.11, which almost certainly was going have a few bugs and issues being the initial release of a new OS.

After a few attempts at trying to fix the issue using Google, it seemed that although I wasn’t the only one being stung by this error message, it also wasn’t particularly wide spread. A recurring theme was the use of multiple Volumes on OSX, where the ‘/Users/’ directory was not where the App Store expected it to be.

Qora (#blogtober 28)

infrequent oscillations — Wed, 28 Oct 2015 21:46:15 GMT

Qora is a bit Sword & Sworcery but chunkier. (Excuse the window decorations - I’m having to do this in a coffee shop without FRAPS because Spectre is going to make me too sad to play any games when I eventually get home after TWO DAMN HOURS of RELENTLESS DRIVEL.) I’d like to point out here is that there’s no voice acting (rapidly becoming a pet hate of mine) in Qora - which means you can have lots of nice lovely textual conversations.

Prison Architect (#blogtober 27)

infrequent oscillations — Wed, 28 Oct 2015 02:07:09 GMT

I went into Prison Architect expecting a subtle, brooding, sneaky moral choice experience much like Papers Please (highly recommended, btw) … and was promptly set about the head by a two by four as the first mission was to build an execution chamber and then execute someone. Way to go with the subtle and brooding there, champ. It’s just grim. But hey ho, maybe that’s just the first “shocker” mission to loosen you up for some lovely subtle … what’s that?

Links - October 28, 2015

Avy Faingezicht — Wed, 28 Oct 2015 02:00:00 GMT

Several comments today: First, if you are reading the AVC posts, make sure to read them together. I think the analogy is flawed, but the underlying idea of "collecting the economic surplus of a revolution" is spot on. Second, I'd like to thank my friend Yoav for pointing out Sapiens. I had skipped the EconTalk episode until he called it the best book he's ever read. I trust his recommendations, even though he doesn't trust mine. Lastly, Politico's website is kind of annoying, but their article about transportation-oriented development in Evanston was super interesting. Having lived there for a good chunk of my life, I can tell first hand that Evanston doesn't feel like a suburb. In fact, when discussing this article, I said "Evanston is actually part of the city, it is basically Chicago" and the response I got was "the fact that you think so means that the policy was successful." Guess so.

[Computing] Why I don't like smartcards, HSMs, YubiKeys, etc.

devever.net/~hl — Wed, 28 Oct 2015 02:00:00 GMT

This is a rant about smartcards with some digression into ranting about the integrated circuit industry.

Copyright, site policies and legal info

Welcome to Marco Fioretti's website! on Stop at Zona-M — Tue, 27 Oct 2015 17:56:59 GMT

Cookie Policy Copyright and Terms of Use Privacy Policy Miscellaneous Legal Info Cookie Policy PREFACE: the European Union saw an actual and very, very serious problem, that is people continuously tracked online in many ways, including “cookies”… and “fixed” it in a terrible way. Italy, where I am based, made it worse. Instead of mandating the addition of cookie management functions in all the very few Web Browsers you may use, it decided to force every website you visit to annoy you with “cookie information” info banners as follows (see the service request below!

Going native with om.next

David Mohl — Tue, 27 Oct 2015 14:47:46 GMT

I mentioned already earlier that I believe om.next will be the future. Not particular om itself but the idea it implements: demand driven applications. The component declares what data it needs and the server is just there for delivering it. Components don’t want to coordinate 20 ajax requests just to gather all the data they need to operate. Components also don’t need all of the data your API provides in most cases, so why even the need to download all of it?

These concepts make sense as the next iteration of server / client architectures but you know where they fit even better? Mobile Apps!

Especially when load time is critical and data volume limited, downloading big chunks of REST json data can get very annoying. We can’t just pump all the jsons our website uses into an app and expect it to be as performant as the web counterpart.

So, wouldn’t it be amazing if our app just says “I need the list of available books, but only the title and the image url and nothing else” and it will get exactly that and nothing more?

With next around the corner I decided to give it a try with react native. cljsrn is already bundling a good amount of resources around the topic clojurescript and react native and natal and ambly already work great for legacy om, so let’s make it work for om.next!

A stoney path

After poking a bit around with next and natal, I quickly ran into a deadend because of it’s dependency to gdom and ReactDOM. Instead of calling React.render, om.next relies on React 0.14 which splits the rendering logic from the component. Here we’re calling ReactDOM.render instead, which, well, relies on a DOM.

Luckily after a short chat with @dnolen he promptly stripped out ReactDOM (and gdom as well) and added functionality that allows us to pass a :root-render and :root-unmount function into the reconciler. Whatever we specify here will get used to render and unmount components.

Reading through the react native source made it clear that 0.14 has not been adapted yet and the rendering and unmounting logic is still inside React. To plug this into om.next, all we have to do is

:root-render #(.render js/React %1 %2)
:root-unmount #(.unmountComponentAtNode js/React %)})

…where js/React is the required react-native library.

Another thing that becomes clear is that React Native doesn’t render components into DOM references. Instead, it uses a number that references back to a component. With the first one being 1, we know exactly what where we have to add our root to:

(om/add-root! reconciler my-component 1)

Wrapping up

The full code looks something like this:

;; Need to set js/React first so that Om can load
(set! js/React (js/require "react-native/Libraries/react-native/react-native.js"))

(ns future-next.core
  (:require [om.next :as om :refer-macros [defui]]))

;; Reset js/React back as the form above loads in an different React
(set! js/React (js/require "react-native/Libraries/react-native/react-native.js"))

;; Setup some methods to help create React Native elements
(defn view [opts & children]
  (apply js/React.createElement js/React.View (clj->js opts) children))

(defn text [opts & children]
  (apply js/React.createElement js/React.Text (clj->js opts) children))


;; Set up our Om UI
(defonce app-state (atom {:app/msg "Hello om.next!"}))

(defui WidgetComponent
  static om/IQuery
  (query [this]
         '[:app/msg])
  Object
  (render [this]
          (let [{:keys [app/msg]} (om/props this)]
            (view {:style {:flexDirection "column" :margin 40}}
                  (text nil msg))
            )))

 (defn read
   [{:keys [state] :as env} key params]
   (let [st @state]
     (if-let [[_ v] (find st key)]
       {:value v}
       {:value :not-found})))

 (def reconciler
   (om/reconciler
    {:state app-state
     :parser (om/parser {:read read})
     :root-render #(.render js/React %1 %2)
     :root-unmount #(.unmountComponentAtNode js/React %)})
   )

(om/add-root! reconciler WidgetComponent 1)

(defn ^:export init []
  ((fn render []
     (.requestAnimationFrame js/window render))))

I submitted a PR to natal and once that’s merged in, all you have to do is natal init :)

Can you believe that all of this happened in just 1.5 days?

Time Travel

Avy Faingezicht — Tue, 27 Oct 2015 02:00:00 GMT

This week's episode of EconTalk opens with an amazing quote, which I felt was to good not to share: > It took only 26 minutes for Anna to push Iris into the hands of the nervous but focused medical student who received her. Twenty-six minutes sounds like a short time for delivery. And it is. And I will argue that the trip that Iris made that night was not a 26-minute trip down a few inches of birth canal, but a hundred thousand year journey from a distant past to an alien future. In 26 minutes, Iris traveled from the ancientness of her mother's womb to the modernity of 21st century society. Birth is in essence time travel. > >-César Hidalgo, Why Information Grows During the rest of the interview, César discusses less philosophical, and more concrete topics, including his research on physics, economics, complex networks and data visualization. I had previously played with his Observatory of Economic Complexity project, and I am very much looking forward to reading about his new work on collective memory. I definitely know what book I am reading next. *Image: "Studies of embryos" by Leonardo da Vinci - Licensed under Public Domain via Wikimedia Commons*

Links - October 27, 2015

Avy Faingezicht — Tue, 27 Oct 2015 02:00:00 GMT

Amnesia (#blogtober 26)

infrequent oscillations — Mon, 26 Oct 2015 23:42:14 GMT

Finally getting around to Amnesia and … meh. Games with voice acting really grind my weasels. It’s just such a waste of resources and effort and then all your dialog has to be plotted for voice and BLEUGH. It actually took a while to get to this point - it turns out that you need to turn off “Display Scaling for High DPI” or the screen will be zoomed and the mouse will jank about like Keith Richards double teaming meth with crack.

Links - October 26, 2015

Avy Faingezicht — Mon, 26 Oct 2015 02:00:00 GMT

The Long Dark (alpha) (#blogtober 25)

infrequent oscillations — Mon, 26 Oct 2015 01:00:11 GMT

The Long Dark is like a first-person Don’t Starve - except prettier and more difficult. This is the only game I’ve seen which lets you pick metric (correct!) and 24 hour clock (correct!). Confusingly, even though the “Feels Like” went from -20º to -15º whilst I was playing, my temperature was heading downwards faster than everyone’s favourite* blue team. I do like a realistic inventory, though. You get some good scenery.

Training for under 50 min 10k run

xenodium.com @alvaro — Sun, 25 Oct 2015 03:00:00 GMT

Not much training time for an under 50 minute 10k run, but here's an attempt (based on time-to-run's sub-50):

Mon Tue Wed Thu Fri Sat Sun

Oct 26 Oct 27 Oct 28 Oct 29 Oct 30 Oct 31 Nov 1 60 min 30 min 2k @ 4.55/k rest 105 min 2 min rest ✔ (repeat x 3) Nov 2 Nov 3 Nov 4 Nov 5 Nov 6 Nov 7 Nov 8 30 min 30 min 1k @ 4.50/k 30 min 30 min rest 5k @ 4.55/k 90 sec rest (repeat x 5) Nov 9 Nov 10 Nov 11 Nov 12 Nov 13 Nov 14 Nov 15 10k easy 30 min 1k @ 4.55/k 30 min 30 min rest race day 1 min easy (repeat x 3)

Reading a running training plan

xenodium.com @alvaro — Sun, 25 Oct 2015 03:00:00 GMT

A sample from Kona Part 2's comments:

2.5 w/u to 4x(1.25@11.5 w/0.25R@7) to 3x(3.75@10.5 w/0.5R@7) to 2.5 c/d.

Is read from left to right as:

2.5 mile warm up to four times through 1.25 miles at 11.5 miles per hour with 0.25 miles recovery at 7 miles per hour to three times through 3.75 miles at 10.5 miles per hour with 0.5 miles recovery at 7 miles per hour to 2.5 miles cool down.

Why Medium?

Avy Faingezicht — Sun, 25 Oct 2015 03:00:00 GMT

This week, I was surprised to see a piece by an Amazon SVP on my feed. What The New York Times Didn’t Tell You showed up months after a *"scathing, blistering, brutal and harsh"* story about the company appeared on the front page of the New York Times mid August. Other than a leaked internal memo from Jeff Bezos, in which he disowned the Amazon portrayed in the story, there was no official response. Cue Jay Carney, and his post. As SVP of Global Corporate Affairs, and the ultimate spokesperson at the Seattle giant, Mr. Carney's job is to keep Amazon's name in good standing. His post and its contents were expected, even if a few weeks late. No surprise there. What seemed remarkable to me is that Carney, and by extension Amazon, decided to publish their account not on their own website, nor on a renowned journal, but on Medium, of all places! Hours later, the New York Times responded with 1000+ words, not on their own site, but on Medium, to which Carney countered, again on Medium. We're not talking about a couple of bloggers with some following having a back and forth, but the Executive Editor of the NYT, and the Senior Vice President of a tech giant engaging in a pissing contest online. On Medium! Eventually, things moved back to a more traditional, um, medium. The release was clearly a calculated move. Publishing it, not on their own site, as an appeal to their massive user base, and not in any prestigious and high-readership publication, was not a decision taken lightly. At a place like Amazon, there must have been at least some data crunched behind this. As usual, Ben Thompson hit the nail on the head, noting that "the placement helped guarantee attention from a certain segment of the public without tying it too explicitly to Amazon," and then went on to describe a broad view of the future of journalism. The more important question, in my opinion, was left unanswered. Why Medium? For starters, the post was not just an exercise is targeting techies. Although Medium is still a platform mostly frequented by a younger, tech-centered demographic, it is gaining breadth quickly. This makes me think that the goal was not to reach the tech crowd specifically, who would be more likely to find it in a narrower publication. Back in May, Medium boasted 25M monthly users. They can't all be tech bros. Obviously, Amazon has a great channel for reaching as many people as possible: Its ever-evolving front page has an audience orders of magnitude larger than whatever Medium dreams of providing. Much like the banners asking for donations on Wikipedia every few months, a rightly placed banner on the Amazon front page would surely make the world listen. However, this would come at a cost. An announcement with such a negative connotation would bring unwanted attention, and distract its customers from ordering more Elsa Dolls. Distribution for the sake of distribution, and reach for the sake of reach, were not the driving force here. Interestingly, Amazon's post is never clear on whether it presents an official version, or just that of an employee with a strong opinion. As the New Yorker points out, during his time at the White House, Carney also opted to tone down the "official-ness" of press releases. Now, he is applying the same strategy at Amazon. Even with the article's first person plural voice, a distracted or careless reader could easily let the extent of its authorship slip. And, let's face it, thanks to Medium's UI, which focuses on the content and nothing else, uninformed readers (which sadly are a majority) would not tell: Carney never states his role in the text, and the reader does not get to see his identity until the very bottom of the page. Had the goal been to publish an official stance, this would have been a really bad way to do it. Medium allowed for more vagueness than a traditional publisher would have, and I believe that was the main objective when selecting how to release the piece. Everyone else is asking, why publish this at all, and maybe more importantly, why now. I have not been able to come up with any good explanations. All I know is Ev Williams probably had a great day this past Monday. *This article was cross-posted on Medium*

2015-10-25

ho.dges.online — Sun, 25 Oct 2015 03:00:00 GMT

That perfect moment when you realise you have one over on your big brothers…

Gravity Ghost (#blogtober 24)

infrequent oscillations — Sat, 24 Oct 2015 23:26:26 GMT

Like one of those old Flash games where you had to propel your penguin between a bunch of planets to hit the black hole1, you have to leave one planet, get the star, and go through the door. This is why I keep my hair violently short - I’m fed up of being harassed by animal spirits for free rides. My luscious locks have snared me a space bunny!

Gathering Time Series Data

denvaar's website — Sat, 24 Oct 2015 08:40:09 GMT

A brief description of an efficient solution to gather new time series data.

A Wizard's Lizard (#blogtober 23)

infrequent oscillations — Fri, 23 Oct 2015 22:48:08 GMT

A Wizard’s Lizard – which sounds like a dirty, dirty euphemism – seems to be heavily inspired by another game. See if you can guess which one. PLOT. This screen, dark though it may be, contains all the clues you need to know about which game heavily influenced this one. It all seems to have gone a bit Dark Souls here - I died and now I’m unhollow? Or hollow-but-angel?

Find binary in PATH using python

xenodium.com @alvaro — Fri, 23 Oct 2015 03:00:00 GMT

import distutils.spawn
print distutils.spawn.find_executable('git')

/usr/bin/git

Links - October 23, 2015

Avy Faingezicht — Fri, 23 Oct 2015 03:00:00 GMT

Antichamber (#blogtober 22)

infrequent oscillations — Thu, 22 Oct 2015 22:25:07 GMT

There’s probably some trick to Antichamber that I’m missing because the only non-obvious puzzle part I managed to figure out were the dreamcatchers and even then the wall had to hint heavily at it. Which is a shame because it looks lovely. Probably one for the philosophy students.

Tools for Cleaning Up Messy JavaScript

Pete's Points — Thu, 22 Oct 2015 19:38:52 GMT

In my consulting work, I encounter a pretty high quantity and diversity of codebases as client projects flow in and out. For many reasons, it is very common for a project to get increasingly messy with time, and often by the time it drops in my lap, it's a certifiable mess.

Dealing with a mess can be a huge drag and productivity drain as even the simplest debugging investigation is rife with confusion, duplication, misdirection, etc. Here's a few quick tools tips to help get things organized and clean again.

esformatter

If the overall code style is highly inconsistent or hard to read, it might make sense to hit it with a giant esformatter hammer and just force everything into a single style. The go programming language does this language-wide via gofmt. Keep these tips in mind:

javascript is interpreted not compiled so if auto-formatting goes wrong and you don't have enough code coverage in unit tests, you might not discover problems until production
- minimize this risk by autoformatting in small batches and making sure any tests you have continue to pass
esformatter and similar tools are still pretty young and could have bugs that change your code in bad ways
- always carefully inspect the diffs before committing automatic changes
Do the formatting in small batches and group changes into a large number of small git commits
- after every change, confirm via unit tests if you have them and eslint that the code is still not obviously broken
- keeping the git commits granular will enable you to more effectively bisect the code to track down a specific problem introduced by one of these changes
Do all the work on a branch, and do no manual code changes of any type on this branch
- For example, don't mix some manual variable renames with automated esformatter changes
- You want to be able to discard the branch entirely without losing and human-authored changes
Get the project done quickly
- This is a complete recipe for merge conflicts and you'll never get it done doing small bits at a time while active development is happening concurrently. If you need to, just declare a small moratorium on development for a weekend or whatever and get everything formatted and merged before new feature development continues

eslint

I covered eslint in some detail in my previous post. Once you've got the code reasonably formatted and consistent, throw eslint at it to get a sense of where bugs and issues may lie. Often times you may be getting hundreds or thousands of errors and warnings. I think focusing on either the most frequent errors (codebase improves the most by fixing these) or least frequent (fixing these can get you to OK on a specific eslint rule quickly) are reasonable approaches. The key thing here psychologically for me is to not get overwhelmed and frustrated and hopeless.

One nice plugin that can help with initial analysis, triaging, and scheduling is eslint-stats which can make it easy to see which problems are most common.

beautify-with-words

True story. A client's codebase was uglified then autoformatted by a previous developer before delivery to the client as "source code". Thus at a glance it looked like source code as it had newlines and indenting, but all the variable names had been minified to single letters. This made it extremely difficult to read. One tool that helped us gradually get back to sanity was beautify with words which finds all those 1-letter variable names and generates a longer, unique, pronounceable (but otherwise gibberish) variable name for them. After that you can easily find and replace all once you understand what an appropriate semantic name for the variable is.

grasp

There's a really cool utility called grasp that parses javascript into an abstract syntax tree and allows you to programmatically alter that tree, then generate new source code. Lots of potential for interesting uses here, but so far I've only really used it for the use case of syntax-aware search and replace. The problem with generic text editor search and replace for variables is it will also change that name within a string or a comment or embedded within another word, etc. The bottom line is careless find/replace in a generic text editor can break your code. With grasp, you can tell it to replace just an identifier and it really knows what that means. For example, to rename a variable from user to account, we could do:

grasp '#user' --replace account

My workflow with grasp is as follows.

highlight a small but coherent and valid javascript snippet in my text editor
- has to be valid JS to work with grasp. I usually grab an entire function declaration or conditional block
copy it into the clipboard
run grasp in the terminal:
- pbpaste | grasp '#user' --replace account | pbcopy
- this pastes the text into grasp's stdin, and copies grasp's stdout back into the clipboard
back in my editor just paste the results in, replacing the still-selected original snippet

Keep It Clean

Hopefully these tools will help you out in the wild cleaning up messy codebases!

eslint: toward javascript lint nirvana

Pete's Points — Thu, 22 Oct 2015 18:42:56 GMT

The world of javascript code linters is a fertile breeding ground for vociferous debate over unimportant details. It's easy to stray into the bad patterns of arguing with colleagues, playing "curly brace police", and other variants. However, used well with the right mindset and team communication, good code linters and autoformatters can be truly valuable additions to your development workflow and tool chain.

If you haven't yet discovered eslint, it's my pleasure to refer you to the eslint about page. Go read the eslint philosophy. It's the strictly correct thing. eslint is "agenda free" and 100% configurable. Earlier tools starting from the grandparent of Douglas Crockford's jslint, then jshint veered off the rails there and eslint is here to take over.

So getting started with eslint can be as easy as:

npm install --save-dev eslint
echo node_modules >> .eslintignore
PATH=$(pwd)/node_modules/.bin:$PATH
eslint .

The eslint rules have pretty good, clear names. The docs are solid. I wrote my own custom formatter called eslint-formatter-comment which I use for text editor friendliness and rule-disabling convenience.

If you want to role with some of the more prolific npm authors, you can decide to start with the @feross/standard configuration by doing npm install --save-dev eslint-config-standard then adding this to your .eslintrc JSON file:

{
  "extends": [
    "./node_modules/eslint-config-standard/eslintrc.json"
  ]
}

Formatting With esformatter

OK great so eslint has found hundreds of issues with your code. What do you do? Manually fixing them can be very time consuming, tedious, and error prone. eslint now has the --fix command and can automatically fix up your source code for some (but not all) rules. I haven't tried it out yet as when I first started working with eslint that feature didn't exist. But you should give it a try and hopefully its capabilities will continue to expand with time. However, there's another project esformatter that can do most code style fixes (indentation, braces, spacing, semicolons, etc) automatically.

When I found esformatter, tried it, and found it to be by-and-large correct and reliable, it was a godsend. I work on so many different client projects as well as some nonprofit codebases where most developers only contribute on a handful of days. This leads to highly inconsistent codebases in terms of format/style. To be able to just define a .esformatter config and fix up the entire codebase in one fell swoop is really awesome and can be a big morale boost as well. esformatter does have some bugs, but so far they are minor and easy to work around in my experience.

How I Roll With eslint

install the stack
- npm install --save-dev eslint eslint-config-standard eslint-plugin-standard eslint-formatter-comment
set up my .eslintrc with the extends as above
set up my .eslintignore with at least node_modules and whatever other project-specific paths I need to ignore
start running it via just eslint . or a small wrapper script that uses my custom output format

Atom Text Editor Integration

I have the linter and linter-eslint atom plugins installed for good integration with Atom. I can directly spot errors in my code without being visually distracting or overwhelming.

For esformatter I have the Atom esformatter plugin and a ctrl-f keystroke defined:

'.editor':
  'ctrl-. e r c': 'eslint:reload-config'
  'ctrl-. l i': 'eslint:lint'
  'ctrl-f': 'esformatter'

Thoughts on feross/standard

@feross/standard is a bit controversial, but the specific choices it puts forward seem good enough to me, and given there are good tools to just automatically format code to that style, I'm OK with just embracing it. I've got 3 medium-sized projects all using it as the baseline and it's OK. I make it even stricter by adding a few more rules, most notably I'm still strongly in the camp of 80-char max line length, so I enable that rule as well. I certainly would have made some different choices, in particular I think double quotes are much better because JSON, but now that I can just hit ctrl-f and have the correct quotes, it's OK.

Why Do You Even Care?

I have had colleagues chastise me for being picky about code format, saying it's a matter of personal choice and not worthwhile to point out or fix. For me, I find consistent style to be a marker of being detail-oriented and paying close attention. If a developer is sloppy or inconsistent with their code formatting, I'm immediately concerned that maybe they aren't paying careful attention to the actual behavior and semantics of their code either. It's so easy to let unused variables, shadowing, and other common errors slip into your code over time, and automated linting provides an easy fix.

Toward Nirvana

Ideally, I'd want to see a single tool like eslint be able to handle all these tasks:

show errors on the command line
show errors in the editor
automatically fix files in place on the command line
automatically fix files in place in the editor

eslint seems headed clearly in this direction, but it's not there yet. I would love to see eslint completely subsume esformatter and make it obsolete.

MarkRight - a markdown editor in om+electron

David Mohl — Thu, 22 Oct 2015 13:43:17 GMT

If you follow clojurescript development you might have already heard about the next iteration of om, om.next.

om.next is David Nolens attempt to bring together what big companies like facebook with relay and netflix with falcor are doing at the moment. I will not go into detail but instead of REST, it implements a architecture in which components clearly declare what kind of data they need to operate. A server / router is then gathering that data and passing it back. Instead of multiple REST endpoints, you’ll end up with 1 graph endpoint that understands the client query.

While still in early alpha, I was looking for an excuse to play around with it - or at least a excuse to use it in the most simple case possible. After scratching my head for a bit I came up with a markdown editor. But why the hell would I want to write a markdown editor?

When I have to write markdown, I’m missing a perfect live preview. I know, there are options out there that give you that either webbased or stuff that just let’s you preview what you wrote. The most popular probably being mou.

But I don’t like mou and it looks like they seem to charge for it soon. On top of that, I was eyeballing githubs electron (the thing that atom uses) for a while and wanted to give it a try.

Meet MarkRight

So I played a bit electron and om, created a pull request to descjop on the way and eventually came up with MarkRight.

Markright is like the most simple editor you can imagine but it is this simplicity that I love! It is literally powered by 2 (or 3 if you count root) om / react components and all they do is piping your input text in real-time through marked. It was fun to give codemirror a spin as well!

I was really surprised how easy it was to just plug these different components together. The open source community did an amazing job with building and maintaining these cool toys!

Anyway, here’s how it looks:

There’s not much to it yet. You can save files, open files and - well - write markdown. There’s a lot of things to do but if you like the idea, please feel free to contribute at github :)

Java 8 parallel streams API

Studiofreya SSG Site — Thu, 22 Oct 2015 09:51:16 GMT

Parallel programming is a very difficult, but sometimes a necessary task. Modern computers contain multiple cores and any complex algorithm that is not performing parallel processing is not exploiting the full potential of the hardware.

In this Java 8 Streams tutorial we will show you how to convert a Stream to List using parallelStream(), filter() and collect(), how to convert an Iterator to Stream and count strings.

The Streams API introduced in Java 8 makes it a lot easier to implement parallel processing. The API does a lot of complex low level processing itself.

Querying AD Groups with PowerShell

Daniel Little Dev — Thu, 22 Oct 2015 05:33:15 GMT

Getting the list of the AD groups if nice and easy in PowerShell. All the CmdLets are located in the module which you might not have…

Indonesia travel bookmarks

xenodium.com @alvaro — Thu, 22 Oct 2015 03:00:00 GMT

Malaysia travel bookmarks

xenodium.com @alvaro — Thu, 22 Oct 2015 03:00:00 GMT

Mongolia travel bookmarks

xenodium.com @alvaro — Thu, 22 Oct 2015 03:00:00 GMT

Running bookmarks

xenodium.com @alvaro — Thu, 22 Oct 2015 03:00:00 GMT

Media player bookmarks

xenodium.com @alvaro — Thu, 22 Oct 2015 03:00:00 GMT

cmus, a small, fast and powerful console music player for Unix-like OS.
mps-youtube.
mpv (a fork of mplayer2 and MPlayer).
Multimedia on Linux Command Line: wget, PdfTK, ffmpeg, flac, SoX.
PLEX (stream your media everywhere).
Soul – A language and IDE for audio coding .

Tabletop Bookshelf

june.kim — Thu, 22 Oct 2015 03:00:00 GMT

Links - October 22, 2015

Avy Faingezicht — Thu, 22 Oct 2015 03:00:00 GMT

The Adventures of Van Helsing (#blogtober 21)

infrequent oscillations — Wed, 21 Oct 2015 22:07:44 GMT

Abysmal. Awful. Arse. Atrocious. I’m beginning to associate cut scenes and voice acting with games that should be put into the bin before they’re even made. Also - picking 1440x480 as my screen resolution and only offering 1366x768 as the best option (“This change will take effect when the game restarts”) is a guaranteed deletion. Sorry chaps, you got one chance and you blew it.

Kyoto travel bookmarks

xenodium.com @alvaro — Wed, 21 Oct 2015 03:00:00 GMT

Eating My Way Through Nishiki Market, Kyoto | Ever In Transit.
Fushimi Inari Temple: 4KM mountain trail lined with bright orange shinto gates.
Golden Pavillion (macha and biscuits at tea house).
Hakone (day trip for hot baths), see Yuryo spa. Also the pirate boat.
Kibune.
Kinkakuji Temple.
Kiyomizu Temple.
Kuramadera Temple & Kibune Shrine: More peaceful shrine.
Kyoto itineraries.
Kyoto travel tips (doc).
Kyoto walking maps.
Kyoto's train station itself.
Nijo-Jo’.
Nishiki Market: Awesome market. Some say better than Tsukiji.
Origami master and school http://www.orizurusalon-yume.com info@orizurusalon-yume.com
orizurusalon yume (origami).
Ryokan ("kaiseki" meals).
Ryōan-ji (竜安寺).
Sanjusangendo.
Things to do in Kyoto, Japan.
Walking courses (Google maps).
WaRaiDo Nighttime tour.

Links - October 21, 2015

Avy Faingezicht — Wed, 21 Oct 2015 03:00:00 GMT

Programming Arduino Uno (ATmega386P) in assembly

personal code attic — Wed, 21 Oct 2015 03:00:00 GMT

After reading various sources on the topic I've written notes, and a sample project, to illustrate a working setup for the task. Also tried to go a little more in depth in certain parts, to clear up some confusion I've dealt with when researching the topic.

Naissancee (#blogtober 20)

infrequent oscillations — Tue, 20 Oct 2015 21:51:59 GMT

Naissancee is lovely in a style sense but since I’ve already lost the tutorial droid within the first 5 minutes, got stuck in a room I can’t get out of without dying -and- being returned to a point two steps before a level load, I’m not going to bother playing it any more. Just enjoy the pictures.

Links - October 20, 2015

Avy Faingezicht — Tue, 20 Oct 2015 03:00:00 GMT

Crawl (#blogtober 19)

infrequent oscillations — Mon, 19 Oct 2015 21:11:10 GMT

I -think- Crawl is some kind of retro-styled take on Dark Souls but … I didn’t really have much of a clue what was going on. Maybe with more reading of instructions and tips it would make sense. Everyone worships Teok, right? Pretty sure I’ve just had my innards tenderised here. It does have a nifty death animation. And then there’s something about upgrading monsters based on how much wrath you have?

The Leftovers: Season 2

🦉 olshansky 🦁 — Mon, 19 Oct 2015 19:33:10 GMT

Olshansky's review of The Leftovers: Season 2

Links - October 19, 2015

Avy Faingezicht — Mon, 19 Oct 2015 03:00:00 GMT

RocketsRocketsRockets (#blogtober 18)

infrequent oscillations — Sun, 18 Oct 2015 20:42:36 GMT

The visuals are nice but it zooms in and out like a pumping weasel is at the controls and seriously - if I’m a rocket ship, left means “turn left”, it NEVER means “fly left”. Shame, really, because the visuals are lovely.

Writings Reborn

Writings of Urda — Sun, 18 Oct 2015 03:00:00 GMT

After a long hiatus, Writings of Urda returns!

Capsule (#blogtober 17)

infrequent oscillations — Sat, 17 Oct 2015 21:29:00 GMT

Capsule might be a contender for peak game of #blogtober. Atmosphere (literally), tension, and an actual story conveyed by recovered emails rather than garish cutscenes, uncanny valley facial animation, and terrible dialog. You have to get your capsule to the next waypoint without running out of oxygen or power because bad things happen when you run out of either. Your only help is a sonar ping that will (slowly) identify what’s around you.

How I build, deploy, and run Spark

Home on The Log Book of Manuel Kießling — Sat, 17 Oct 2015 18:13:00 GMT

I’m currently tinkering with Spark for my side project JourneyMonitor. The goal is to extract useful metrics from the Selenium runs executed by the platform. To do so, I’m currently in the process of creating a new Analyze component. I want to build the Spark setup and the jobs using Scala 2.11. Therefore, I had to compile my own version of Spark 1.5.1, put it onto the systems, and run a cluster from that.

Get Emacs to gather links in posts

xenodium.com @alvaro — Sat, 17 Oct 2015 03:00:00 GMT

Comments in posts can be a great source of recommendations. Here's a way to extract post links using Emacs and enlive.

(require 'enlive) ;; https://github.com/zweifisch/enlive
(require 'org)

(defun ar/input-clipboard-url-or-prompt ()
  "Return a URL from clipboard or prompt user for one."
  (let* ((clipboard (current-kill 0))
         (url (if (string-match "^https?://" clipboard)
                  clipboard
                (read-string "URL: "))))
    (unless (string-match "^https?://" url)
      (error "Not a URL"))
    url))

(defun ar/url-fetch-anchor-elements (url)
  "Fetch anchor elements in URL as list of alist:
\((title . \"my title\")
 (url . \"http://some.location.com\"))."
  (let ((elements (enlive-query-all (enlive-fetch url) [a])))
    (mapcar (lambda (element)
              `((title . ,(enlive-text element))
                (url . ,(enlive-attr element 'href))))
            elements)))

(defun ar/url-view-links-at ()
  "View external links in HTML from prompted URL or clipboard."
  (interactive)
  (with-current-buffer (get-buffer-create "*links*")
    (org-mode)
    (view-mode -1)
    (erase-buffer)
    (mapc (lambda (anchor)
            (let-alist anchor
              (when (and .url (string-match "^http" .url))
                (insert (org-make-link-string .url
                                              .title) "\n"))))
          (ar/url-fetch-anchor-elements
           (ar/input-clipboard-url-or-prompt)))
    (delete-duplicate-lines (point-min) (point-max))
    (goto-char (point-min))
    (toggle-truncate-lines +1)
    (view-mode +1)
(switch-to-buffer (current-buffer))))

UPDATE(2019-04-13): Refreshed post with latest code from my init. Thanks to Gĳs for pinging.

Grounding, part 3: Longer explanations

D13V — Sat, 17 Oct 2015 03:00:00 GMT

No matter how good we are at explaining, we cannot directly ground all concepts for everybody.

In a video interview, Richard Feynman explained how he thinks about physics. One question asked him to explain how electro-magnetic forces work. He explained that they exist in the form they do. There is no simpler explanation to the electro-magnetic forces than that, he said.

Can we say electro-magnetic forces worked like an everyday object? He gives a counter example. Both magnetic forces and rubber bands can attract things together. He argues that such explanation is more confusing than helpful. That rubber bands depend on electro-magnetic forces to stay together and attract. But these forces are exactly the things we were trying to understand in the first place. We are back where we started from. We have created a vicious circle of reasoning, and we trying to step on top of our head.

A circular explanation never reaches solid ground, and is ineffective in explaining the purpose.

https://youtu.be/wMFPe-DwULM?t=6m2s

If we want to help the other person understand we can take a detour. Instead of giving a direct answer, we can teach them solid physics and mathematics first. Once they have understood the basics, we can introduce the concept.

Without science foundation, the other side has no common ground with the nature of magnetism. Any explanation would be futile, or even worse - misleading and backfiring in the future. It is up to us to find out what they know or don’t know already that can relate to the idea and use it to explain it. We need to adjust our message, and in the case when there is no common ground… we have to build it.

We can only keep a handful of things in our working memory at a time. This limits which things we can explain without the other side losing interest. Still, keeping them interested along the way, makes our job of explaining so much more interesting. We still need to use clear and non-misleading examples and logic. But we also need to provide incentives for other person to put the effort.

For concepts like this, taking shortcuts in explanation will make them appear magical. There will always be a hole their comprehension. It is inevitable that we will need to cover this hole with a trust-me-this-is-how-it-works. We can gradually fill parts of the hole with clear, grounded explanation. But we need to be careful not to create a lot of small and hard to fill holes.

Persistent Login (the "remember me" checkbox)

Databases & Life — Sat, 17 Oct 2015 03:00:00 GMT

It is a requirement of many customer projects of mine to have a "remember me" or "keep me logged in" checkbox on the log-in form. I am not aware of any standard approach to the subject; I suspect every company invents their own (and then blogs about it?) Here's what I do when asked to implement this feature, as online on art.world.

Here are the requirements as I see them:

Users should be able to use the “remember me” function from one computer (e.g. home) but not from others (e.g. internet cafe)
Users should be able to use the “remember me” function from as many computers as they like (e.g. home, their phone, ..)
Typing the user name, password and “remember me” should remember only this user name and password; thus, if they change their password using another computer, this “remember me” should stop working.

Here are some thoughts on various options:

Human Resource Machine (#blogtober 16)

infrequent oscillations — Sat, 17 Oct 2015 00:05:18 GMT

TL;DR: GLORIOUS. Buy it now. Even if you don’t like the game, Tomorrow Corporation deserve all the money they can get to keep making lovely sweet games for ME. I’m more than reasonably sure this is going to be Peak Game for #blogtober. After the surprising depth of Little Inferno, Human Resource Machine was a no-brainer pre-order and here it is. I think the third one bears a striking resemblance to my current form.

Continuous Integration for Lua with Travis

Separate Concerns — Fri, 16 Oct 2015 17:30:00 GMT

Travis is a Continuous Integration service which is free for Open Source projects and has very good GitHub integration. We will see how to use it for your Lua projects.

Your test suite will work well with Travis as long as executing it returns 0 on success and nonzero on failure. If you use plain Lua assertions, it is already the case. If you use a test framework, make sure that it works that way. I have added a helper to cwtest for that purpose.

Travis does not support Lua out of the box, but using it with Lua projects is not hard because moteus has done all the hard work for you. You just have to clone this repository and copy the .travis directory to yours.

After that, you only have to write a single YAML file, .travis.yml. For example, here is the one I wrote for Haricot.

Most sections should be self-explanatory. install is where you set up your dependencies. The first line calls moteus’ script which lets you use Lua and LuaRocks. A separate build and test run will occur for every Lua version declared in matrix; you can comment some lines there if you do not want to test some Lua versions. For Haricot I need Beanstalk running in the background so I start it in before_script. script is where you run your actual tests.

To enable Travis, sign up, allow Travis to access your GitHub account, then go to your profile and flip the switch for your repository:

After that, commit .travis and .travis.yml and push to GitHub. It will trigger a test build, and so will every subsequent commit. If you want, you can now add a badge with the status of your build to your README or your project’s home page.

OS X Finder Woes

bastibe.de — Fri, 16 Oct 2015 12:51:00 GMT

The Mac. It used to be the most streamlined, thought-through general computing device on the market.

Even it's file management used to be top-notch. There were many cool little touches. One particularly useful feature was the Proxy Icon--if a window displayed a file's content, that file's icon would show up in the window's title. And you could drag that icon directly onto a thumb drive or email, without having to use the Finder. But the Finder, too, had many neat little features. I loved the fact that when you renamed a file in an alphabetically sorted file list, Finder would not immediately re-shuffle it to its new location, but would wait half a second before doing so. When renaming multiple files, this was really useful, since you could go through them one by one and rename them, simply by pressing arrow keys and return.

But as you might have guessed from my use of the past tense, these golden days are gone. The Finder used to know a JPEG from a ZIP regardless of file extension. Now it doesn't any more. The Proxy Icon is still draggable, but it will create an alias instead of a copy--perfectly useless on a thumb drive or in an email.

And with the newest version of OS X, El Capitan, they finally blew it for me. Before, even though the Finder inexplicably never had the ability to cut and paste files, you could always install programs like TotalFinder to fix that. Not so with El Capitan. The Finder now is holy land, and can not be touched any more by third parties. So no more cut and paste, no more un-hiding system files. No more side-by-side Finder tabs. And brand new with El Capitan as well: No more waiting after renaming. Now, when you rename a file, it is immediately re-sorted to its new position, thus making renaming multiple files terribly inconvenient.

So, good bye OS X. I updated my work laptop first, and I regret it. I never regretted an OS X update before. My home machine is not going to get the update. It is honestly sad to see my once-beloved Mac platform becoming worse and worse and worse with every new release.

UX toolbox bookmarks

xenodium.com @alvaro — Fri, 16 Oct 2015 03:00:00 GMT

My talk at Percona Live 2015

Nicola Iarocci — Fri, 16 Oct 2015 03:00:00 GMT

The slides of my talk at Percona Live 2015 (Amsterdam) are online. It was titled “MongoDB and REST APIs a Match Made in Heaven” and it was meant as an introduction to Mongo, REST principles and the Eve python framework.

Overall it has been a pleasant experience, although I found that splitting 300 attendees through seven concurrent tracks ultimately led to too much fragmentation. People often found themselves with 2-3 interesting talks all happening simultaneously, and just had to pick one.

A Virus Named Tom (#blogtober 15)

infrequent oscillations — Thu, 15 Oct 2015 22:28:41 GMT

Remember Pipe Mania? Well, A Virus Named Tom is basically a tarted up Pipe Mania with a plot. See? Plot. I’m stuck here because all the “encrypted” (aka “hidden”) tiles are straight pieces and I can’t figure out how everything connects up. Plus the rotation of the pieces is done by dragging the little robot around the border and trust me, that gets pretty old pretty damned fast as a control mechanic.

Forget fablabs and makerspaces! Who ELSE is promoting Digital DIY in Europe?

Welcome to Marco Fioretti's website! on Stop at Zona-M — Thu, 15 Oct 2015 13:11:21 GMT

As some of you may already know, these days I also work in the H2020 research project “Digital DIY”, a world of which 3D printing is only the most fashionable part, but not the biggest, nor the most important. Among other things, right now we’d need to know something that is pretty hard to discover without _local (meaning: yours!) _assistance, because it is “hidden” behind many different languages and layers of burocratic structures and inertia:

Zippered Pouch

june.kim — Thu, 15 Oct 2015 03:00:00 GMT

Links - October 15, 2015

Avy Faingezicht — Thu, 15 Oct 2015 03:00:00 GMT

The Intelligent Investor

🦉 olshansky 🦁 — Thu, 15 Oct 2015 03:00:00 GMT

Olshansky's review of The Intelligent Investor by Benjamin Graham

Monitoring with Bosun

Brightball Articles — Thu, 15 Oct 2015 03:00:00 GMT

Bosun is a monitoring and alerting system developed by the good folks at Stack Exchange, then open sourced for the rest of us. It’s written in Go, meaning its monitoring agents can run anywhere that Go can drop a binary… which is just about everywhere. So what exactly does it do and how does it compare to the likes of New Relic, CloudWatch, Nagios, Splunk Cloud, Server Density, and other monitoring tools?

Dustforce (#blogtober 14)

infrequent oscillations — Thu, 15 Oct 2015 00:40:01 GMT

You have to sweep up leaves. Sadly without actually using your brush - a simple run through is sufficient to magically displace them into a nether dimension. The jumping and controls are horribly floaty and syrupy and laggy and ugh. AND it not only doesn’t have controller support by default, it let me accidentally assign the same joypad button (“pad0_dir2” is also a tremendously horribly label) to two different directions.

Links - October 14, 2015

Avy Faingezicht — Wed, 14 Oct 2015 03:00:00 GMT

NextPlayer: A Node.js module for managing player turns

Thomas Hunter II — Wed, 14 Oct 2015 03:00:00 GMT

Torchlight (#blogtober 13)

infrequent oscillations — Wed, 14 Oct 2015 00:50:36 GMT

A hacky-slashy adventure RPG - kinda like Geneforge but with 90% of the budget spent on 3D modelling and voice actors instead of plot and story. Although it does have a rudimentary plot. It does look awfully pretty - but I can see myself playing Geneforge in preference to this just because it’s much more satisfying than this melange of mishmash.

Python Fabric

Thought Eddies — Tue, 13 Oct 2015 10:26:00 GMT

Python Fabric

My Dream Editor

David Mohl — Tue, 13 Oct 2015 10:05:17 GMT

Today, let’s talk about editors. Why? Because currently there are more editors than anything else. Microsoft decided to launch an editor, github decided to launch an editor - heck, even facebook suddenly invests into editors. We have closed source ones, open source ones, big full powered IDEs and of course the old school family.

The good thing, I actually like playing with editors. Sometimes I feel like just opening a new tool and test it out. Because of that I discovered SublimeText from NetBeans / PHPStorm. Because of that I also switched from SublimeText to VIM and later back to intellij. I love to constantly improve my workflow and find new ways to get even more productive.

The annoying thing about testing editors is that well, once you hit VIM you are pretty much in a dead end. From then on each and every editor has to have VIM keybindings. If it doesn’t, you can’t use it. Naturally I spent a ton of time with VIM - the best vim emulation out there. SublimeText and Atom just couldn’t cut it. They add a good amount of sugar into the mix, sugar I really like - but their VIM plugins are just not good enough (yet). So I kept going my way with vim, carefully tweaking my good old .vimrc as days come by.

To my big surprise, the editor that won me by storm wasn’t vim and it wasn’t neovim either. It was something that made me drop my beloved .vimrc without hesitation, something I thought I would never even touch - emacs. Yeah, emacs made me drop all other editors in the year 2015. Ridiculous, right? So what’s going on?

Enter spacemacs

spacemacs is an amazing pre-configured emacs made for people like me! VIM users!. It uses the power of evil, the arguably best VIM emulation out there and builds a layer of highly productive keybindings on top. Here’s how it works:

In spacemacs, each command is triggered by the spacebar and follows a very innovative mnemonic. spc and b for example is showing you actions on the current buffer. spc and p executes project commands.

The really cool thing though comes from it’s layer system. spacemacs has a ton of contributed layers that just work. For example without researching anything, I just enabled the react layer and suddenly I had full jsx support, highlighting, linting with eslint and a ton of keybindings! Just like that! Why? Because someone who actually uses react implemented it this way, the way it should be done.

So many strong editors give me a ton plugins and what not without actually knowing the ecosystem of the target language / framework just for the sake of supporting as many things as possible.

And if the plugin is superb, then the base of the editor is usually weak. Look at VIM: We have so many cool plugins but most of them are a big ugly hack in vimscript that works so-so. Emacs on the other hand is hackable in every little corner. It provides a powerful scripting language in form of elisp and has a very active community around it.

I would even argue that most of the layers that ship with spacemacs out of the box are a good amount better than what I hacked together inside my .vimrc.

I was so impressed by it that I decided to port parts of it to sublimetext.

My lovestory with spacemacs begins. Where were you all my life, you beauty of an editor?

What else?

Here’s my ‘mini ranking’ of editors I like:

Not My First Post

Avy Faingezicht — Tue, 13 Oct 2015 03:00:00 GMT

Today, out of curiosity I went back to the first post in the archive of Fred Wilson's AVC. I was surprised by the terseness, and the simplicity. Maybe that's all I need to start writing more. Lowering the bar I set for myself. If Fred could do it, way back in September 2003, why shouldn't I? > I read blogs a lot. And i think they are great. So i am starting a blog. I have no idea if i’ll write a lot in my blog or rarely. I hope its a lot, because i have a lot to say. But we’ll see about that. > > -Fred Wilson Seems like he did have a lot to say. Wilson is one of my favorite bloggers. I read him daily, making his closing statement particularly entertaining in hindsight. Another inspiring case is Benedict Evans, who five years ago was getting only ~200 monthly page views and now gets hundreds of thousands of hits per month. However, his strategy was very different: his first posts are in-depth analytical essays, just like the ones he posts in 2015. I'll aim in between, and settle if I can get even a tenth of their audience.

Links - October 13th, 2015

Avy Faingezicht — Tue, 13 Oct 2015 03:00:00 GMT

It's not that there were no interesting things to read on the internet this past week, its that I was lazy and have not posted them. Here's another catchup post.

spacewΛr

Matt Keeter — Tue, 13 Oct 2015 03:00:00 GMT

Lasers! Video games! Hardware!

Heroes by Adam Grant

ho.dges.online — Tue, 13 Oct 2015 03:00:00 GMT

My childhood hero was Superman. Now, I try not to put people on pedestals – everyone has their Kryptonite. Instead of seeing my role models as heroes, I admire them for specific virtues they exemplify.”

Adam Grant

Mirror Moon (blogtober #12)

infrequent oscillations — Mon, 12 Oct 2015 23:32:54 GMT

Baffling space exploratory puzzle something something. Possibly the tiniest minimalist title screen yet. Looking a bit like a low rent NES-inspired Elite Dangerous here. That’s quickly dispelled once you arrive on the first planet. Except I can’t really say anything about the next few screenshots since it’s probably spoilery. (I will say that I had to resort to walkthroughs because they do a bloody awful job of letting you know what you’re supposed to be doing.

How to use Java varargs

Studiofreya SSG Site — Mon, 12 Oct 2015 22:30:44 GMT

The three dots in a method parameter type is a feature called Java varargs. It was introduced in Java 5 and is formally known as variable arity methods or variadic functions.

Contents

What is varargs

Varargs methods accept zero or more arguments (args) of a given type. Such methods can have a variable number of arguments and thus useful for passing of dynamic data.

The Future of Web Dev - React, Falcor, and ES6

Train of Thought — Mon, 12 Oct 2015 03:00:00 GMT

The future of web application development looks a bit different than what we are all used to. I'll show you how to build a simple full-stack JavaScript app using Node.js on the backend, React on the frontend, Webpack for client-side module support, and Netflix's Falcor as an efficient and intuitive alternative to the traditional REST API.

How to get better at software development?

Nemo's Home — Mon, 12 Oct 2015 03:00:00 GMT

I often get a lot of queries from people asking me about how to get started with software development, and how to get better at it. My replies are almost reaching stock-level worthy of copy-paste now, so I thought I might as well write about it publicly.

What follows is a list of advice I’d give to any person who wants to write software for a living. A lot of it might apply across professions, and a lot of it is tailored to students in universtities. Not everything might apply in your case, YMMV. Take everything with a pinch of salt. Feedback is welcome.

Join a community.

Highly preferable if its an IRL (In-real-life) community rather than just a chatroom somewhere, but even those are preferable over nothing. Communities have this shared sense of learning, that you don’t enjoy anywhere else. Passive learning is something I talk a lot about, and it only happens because of chance interactions that happen in communities. Even online communities work fairly well, and by online communities I mean places like StackOverflow, AskUbuntu, ServerFault, HackerNews, subreddits etc. If you don’t have a physical community near you that you can join, maybe its time to start one?
Contribute to Open Source projects

It doesn’t have to be with your code, or even a large project. Even small javascript npm modules that you might think can be improved deserve some Pull Request love.
Write all code publicly

Your code not being public should be the exception, not the norm. I’ve found putting almost all my code on github fairly liberating. I keep all my OS configuration and a lot of other things on github.
Do tech talks

It doesn’t have to be at a big-name conference, but maybe at a small meetup around you. Good conferences will sponsor your tickets, and as a plus, you get to attend all the talks at that conference for free. Just make sure that you actually do know what you’re talking about, unlike a lot of talks that happen. The level of knowledge expected of a speaker is far more, and as a result if you are the one talking about something, you need to get better at it and understand it better, which is a great way of forcing yourself to learn something.
Stay Updated

Reading Hacker News is a fairly certain way of making sure of that. A person doing PHP development should be aware of things like Composer, HHVM, and perhaps the upcoming changes in PHP7 (They’re awesome). As a technologist, part of our job is to stay updated with trends (no matter how insane the JS framework wars sound). The code you will be writing 5 years from now will be in an entirely different framework than what you are using today. This doesn’t mean that you should start learning the ins and outs of every JS framework, but rather that you should be tangentially aware of developments happening in the space. (For eg, following stable updates of Rails even though you are not a Rails developer).
Learn more languages

I am a proud polyglot, and I very often realize that knowing more than one language changes your style and more importantly your thinking process significantly. For eg, a Ruby programmer will be fairly comfortable with the idea of metaprogramming compared to a PHP developer, and even more so when it might come to DSL (Domain Specific Languages). Similarly, knowing Haskell or Functional Programming in general teaches you a lot of things that you might re-use back in your JavaScript world.
This doesn’t happen unless you know more than one language. Moreoever, its always helpful to have JavaScript as your second language (if you are looking for one), because of its monopoly in front-end development. A lot of technologies (like CORS/JSONP) just don’t make sense unless you understand JavaScript.
Concepts Matter

I was asking people about good interview questions, and one that I really liked was “How do you write an HTTP server using sockets?”. A lot of developers are stuck in this moat of “programming = software development”. And you can’t get over that unless you start thinking in terms of concepts. This is not me trying to get people to become Architecture Astronauts, but me trying to get people to understand how things work. I’ve interviewed people who have no idea about how HTTP works, and in my opinion you can’t really be a web developer without knowing HTTP. A fairly good filter for good web developers is whether they know the ins-and-outs of HTTP. And HTTP is not a programming challenge, but rather a conceptual problem. Similarly, if you work in the frontend, and you don’t know what the Same Origin Policy is, I am not gonna hire you. (“Is it implemented on the browser or the server?” is a another good question). The point I’m trying to make is that you need to get a layer above your language’s standard library and understand how things work. Learning ActiveRecord is awesome, but do you understand how it works?
Ship Products

Doesn’t matter if they are small, or made in a hackathon. As long as its shipped, we’re cool. If its not, come back when you’ve shipped it.
Have side projects

This is slightly harder to do, but far more rewarding. Make sure that your side-project is not something you expect to make money out of, and that it has a fairly reasonable scope. Side projects are an excellent breeding ground for you to try out new technologies, and play around with new languages. Its a really good breakaway from work-things as well, on top of that.
Read technical books

As a start, I’d recommend everything that codinghorror has suggested here and here. There are a lot of good books listed on hackershelf.com as well. My personal favorite is Don’t Make Me Think, which is a book on Web Usability and something I think every developer and designer should be forced to read.

Thanks to Shashank Mehta for discussing these ideas with me and helping me frame this post.

Titan Attacks (#blogtober 11)

infrequent oscillations — Sun, 11 Oct 2015 18:56:33 GMT

Basically a fancy Space Invaders with upgradable weapons and savable hostages a la Defender. They’re confused. Points win prizes, not perfection. Silly aliens.

A new blog

David Mohl — Sun, 11 Oct 2015 14:39:46 GMT

I decided to create another simple blog based on jekyll next to my medium one. Why? let me explain.

Medium is an amazing platform for writing. That’s what works really damn good on it. But writing doesn’t equal blog. Medium is a magazine - a collection of articles and columns from individual writers. When you see a post about ‘How to set up php’ with a big editorial picture and a ton of quotes, don’t you also think that it kind of feels… out of place?

Medium will be the place where I’ll write thought through stuff. Stuff that could go into a magazine. On jekyll (here) on the other hand, I can write anything that is somehow tech related:

Some guide for a new technology? Here.
A article on how to work productively? Medium.
A post on workflow thoughts? Medium.
Code snippets how to set up one of my projects? Here.
My thoughts on a certain technology? Here.
Common mistakes companies make? Medium.

… you get the drill right? And in general, if you subscribe through RSS it doesn’t really matter where the articles come from.

Now there’s just the problem that I really don’t blog much in general…

The Floor Is Jelly (blogtober #10)

infrequent oscillations — Sun, 11 Oct 2015 00:19:24 GMT

Puts me in mind of a weird cross between the creature innards levels of Loco Roco and Bugaboo The Flea. Simple - just bounce your way around the jelly floors to reach the curtains (or lift, later.) This is a cute level - to reach the curtains, you need to properly trampoline yourself with good timing to get a super jump. Just in case the player hasn’t realised this is possible, the little frog on the right is demonstrating.

final bits from fOSSa 2015, from Open Education to Ecology

Welcome to Marco Fioretti's website! on Stop at Zona-M — Sat, 10 Oct 2015 16:48:56 GMT

fOSSa 2015 was such a great conference that I and Wouter Tebbens already wrote four other posts about it (see below). Here are the last bits that are worth sharing but did not fit elsewhere.

Dope

🦉 olshansky 🦁 — Sat, 10 Oct 2015 16:04:59 GMT

Olshansky's review of Dope

Citizen cloud thoughts, after fOSSa 2015

Welcome to Marco Fioretti's website! on Stop at Zona-M — Sat, 10 Oct 2015 12:13:12 GMT

I had (at least) three big reasons to be at the fOSSa 2015 conference, a couple of weeks ago. Two already covered elsewhere and one, “Citizen Cloud: Towards a more decentralized internet?", that deserves its own separate post.

C++ std::remove_if with lambda explained

Studiofreya SSG Site — Sat, 10 Oct 2015 03:00:00 GMT

The programmer has the responsibility to remove or resize the elements from the actual container. The only exception is when remove and remove_if are members of a container, which is the case of std::list remove_if.

National Sh**e Day by Half Man Half Biscuit

ho.dges.online — Sat, 10 Oct 2015 03:00:00 GMT

I try to put everything into perspective,
Set it against the scale of human suffering,
And I thought of the Mugabe government
And the children of the Calcutta railways.
This works for a while.
But then I encounter Primark FM
Overhead a rainbow appears.
In black and white.

#theme/funny

10 Minute Monitor Ambient Light

Justin Garrison's Homepage — Sat, 10 Oct 2015 03:00:00 GMT

Ambient lights help with eye strain and glare from overhead lights. This is a simple light you can install

2015 Predictions

Justin Garrison's Homepage — Sat, 10 Oct 2015 03:00:00 GMT

That’s right, I took “technology” out of the title. Why, because I didn’t want to limit myself. As before,

What It’s Like to Interview for Tech Jobs

Justin Garrison's Homepage — Sat, 10 Oct 2015 03:00:00 GMT

Over the past year I have had great opportunities to interview for positions at companies I respect, and some I didn’t. I wanted to share my experiences to help others looking to go down this job path and some general interviewing tips.

This post originally appeared on Justin Garrison’s blog .

I’ve talked to quite a few companies, including:

Google
Twitter
GitHub
Oculus VR
Disney Animation Studios
Beats by Dre
Tinder
Digital Ocean
DeviantArt
Bass Pro

Each company was different in their approach to interviewing candidates. For most of the companies I didn’t make it past the screening call, but I got job offers from some. Most were for DevOps/Site Reliability Engineer (SRE) positions but for even those that weren’t, the interview process was similar enough to be relevant to my experience. Some companies sought me out, some I applied for, and some were through recruiters I had previously applied through who kept in touch with new opportunities. The companies that found me usually were via LinkedIn or GitHub .

Codebase Cop - A Slack Bot Watching Over Your Tickets

Tafkas Blog — Sat, 10 Oct 2015 03:00:00 GMT

At Project-A we are using Codebase as a project management tool together with its version control. Just as with any other tools you can create tickets and organize them in sprints. Our usual (very simplified) workflow includes: Sprint planning for tickets Priotizing tickets Developer working on tickets Product managers verifying if the tickets were implemented as intended Unfortunately, sometimes your backlog keeps growing and tickets are no longer valid, outdated or, in the worst case, just forgotten.

Spy Chameleon (#blogtober 9)

infrequent oscillations — Sat, 10 Oct 2015 00:30:52 GMT

The screenshots tell pretty much the whole story and I’ve got to get up early. Enjoy.

Rogue Engineers Behind Volkswagen Scandal? So what?

Welcome to Marco Fioretti's website! on Stop at Zona-M — Fri, 09 Oct 2015 15:12:42 GMT

Some recent declarations from VW executives about the Volkswagen scandal are half unbelievable, half totally irrelevant. Seriously.

Advanced Oracle Troubleshooting v2.5 (with 12c stuff too)

Tanel Poder Blog — Fri, 09 Oct 2015 10:58:08 GMT

It took a while (1.5 years since my last class – I’ve been busy!), but I am ready with my Advanced Oracle Troubleshooting training (version 2.5) that has plenty of updates, including some more modern DB kernel tracing & ASH stuff and of course Oracle 12c topics!

The online training will take place on 16-20 November & **14-18 December 2015 **(Part 1 and Part 2).

The latest TOC is below:

/files/AOT25_description.pdf

Seminar registration details:

Travel blog bookmarks

xenodium.com @alvaro — Fri, 09 Oct 2015 03:00:00 GMT

How to connect your PSP on Linux

personal code attic — Fri, 09 Oct 2015 03:00:00 GMT

Common debugging steps you can try if it doesn't get automounted like it should.

FEZ (#blogtober 8)

infrequent oscillations — Fri, 09 Oct 2015 00:52:12 GMT

One of the big indies that I’ve somehow managed to avoid playing until now. I wonder if she has any kerosene Geezer has some cracking industrial strength drugs, it seems. Um, Lament Configuration, anyone? “We have such cubes to show you.” “This will seem like a memory of 2D!” etc. etc. Then the titular headwear arrives from the Lament giant Devil Square and Gomez can move 2D-wise through 3D. (It’ll make sense when you play the game.

Why I love compilers

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Thu, 08 Oct 2015 22:14:00 GMT

Compilers are getting better and better at interpreting the stupid stuff I throw at them. Every once in a while, I manage to surprise them and myself by my stupidity. This time, for your pleasure, a minimum working example—or rather a minimum non-working example:

namespace
{
#include <list>
}

int main(int, char**)
{
  std::list<int> l = {1,2,3,4};
  return 0;
}

Here is what clang++ has to say about this:

$ clang++ --version
clang version 3.6.2 (tags/RELEASE_362/final)
Target: x86_64-unknown-linux-gnu
Thread model: posix
$ clang++ -std=c++11 uh-oh.cc
In file included from uh-oh.cc:3:
In file included from /usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/list:61:
In file included from /usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/bits/allocator.h:46:
In file included from /usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/x86_64-unknown-linux-gnu/bits/c++allocator.h:33:
In file included from /usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/ext/new_allocator.h:33:
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/new:111:7: error: 
      'operator new' cannot be declared inside a namespace
void* operator new(std::size_t) _GLIBCXX_THROW (std::bad_alloc)
      ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/new:113:7: error: 
      'operator new[]' cannot be declared inside a namespace
void* operator new[](std::size_t) _GLIBCXX_THROW (std::bad_alloc)
      ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/new:115:6: error: 
      'operator delete' cannot be declared inside a namespace
void operator delete(void*) _GLIBCXX_USE_NOEXCEPT
     ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/new:117:6: error: 
      'operator delete[]' cannot be declared inside a namespace
void operator delete[](void*) _GLIBCXX_USE_NOEXCEPT
     ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/new:119:7: error: 
      'operator new' cannot be declared inside a namespace
void* operator new(std::size_t, const std::nothrow_t&) _GLIBCXX_USE_NOEXCEPT
      ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/new:121:7: error: 
      'operator new[]' cannot be declared inside a namespace
void* operator new[](std::size_t, const std::nothrow_t&) _GLIBCXX_USE_NOEXCEPT
      ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/new:123:6: error: 
      'operator delete' cannot be declared inside a namespace
void operator delete(void*, const std::nothrow_t&) _GLIBCXX_USE_NOEXCEPT
     ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/new:125:6: error: 
      'operator delete[]' cannot be declared inside a namespace
void operator delete[](void*, const std::nothrow_t&) _GLIBCXX_USE_NOEXCEPT
     ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/new:129:14: error: 
      'operator new' cannot be declared inside a namespace
inline void* operator new(std::size_t, void* __p) _GLIBCXX_USE_NOEXCEPT
             ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/new:131:14: error: 
      'operator new[]' cannot be declared inside a namespace
inline void* operator new[](std::size_t, void* __p) _GLIBCXX_USE_NOEXCEPT
             ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/new:135:13: error: 
      'operator delete' cannot be declared inside a namespace
inline void operator delete  (void*, void*) _GLIBCXX_USE_NOEXCEPT { }
            ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/new:136:13: error: 
      'operator delete[]' cannot be declared inside a namespace
inline void operator delete[](void*, void*) _GLIBCXX_USE_NOEXCEPT { }
            ^
uh-oh.cc:8:18: error: no matching constructor for initialization of
      'std::list<int>'
  std::list<int> l = {1,2,3,4};
                 ^   ~~~~~~~~~
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/bits/stl_list.h:697:9: note: 
      candidate constructor template not viable: requires at most 3 arguments, but
      4 were provided
        list(_InputIterator __first, _InputIterator __last,
        ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/bits/stl_list.h:628:7: note: 
      candidate constructor not viable: requires at most 3 arguments, but 4 were
      provided
      list(size_type __n, const value_type& __value,
      ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/bits/stl_list.h:678:7: note: 
      candidate constructor not viable: requires at most 2 arguments, but 4 were
      provided
      list(initializer_list<value_type> __l,
      ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/bits/stl_list.h:604:7: note: 
      candidate constructor not viable: requires single argument '__a', but 4
      arguments were provided
      list(const allocator_type& __a) _GLIBCXX_NOEXCEPT
      ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/bits/stl_list.h:616:7: note: 
      candidate constructor not viable: requires single argument '__n', but 4
      arguments were provided
      list(size_type __n)
      ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/bits/stl_list.h:655:7: note: 
      candidate constructor not viable: requires single argument '__x', but 4
      arguments were provided
      list(const list& __x)
      ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/bits/stl_list.h:667:7: note: 
      candidate constructor not viable: requires single argument '__x', but 4
      arguments were provided
      list(list&& __x) noexcept
      ^
/usr/bin/../lib64/gcc/x86_64-unknown-linux-gnu/5.2.0/../../../../include/c++/5.2.0/bits/stl_list.h:593:7: note: 
      candidate constructor not viable: requires 0 arguments, but 4 were provided
      list()
      ^
13 errors generated.

Suddenly, everything appears to be broken. No initializer lists even though C++11 is supported? What the what? Let us check what g++ has to add:

$ g++ --version
g++ (GCC) 5.2.0
Copyright (C) 2015 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
$ g++ -std=c++11 uh-oh.cc
In file included from /usr/include/c++/5.2.0/ext/new_allocator.h:33:0,
                 from /usr/include/c++/5.2.0/x86_64-unknown-linux-gnu/bits/c++allocator.h:33,
                 from /usr/include/c++/5.2.0/bits/allocator.h:46,
                 from /usr/include/c++/5.2.0/list:61,
                 from uh-oh.cc:3:
/usr/include/c++/5.2.0/new:112:41: error: ‘void* {anonymous}::operator new({anonymous}::std::size_t)’ may not be declared within a namespace
   __attribute__((__externally_visible__));
                                         ^
/usr/include/c++/5.2.0/new:114:41: error: ‘void* {anonymous}::operator new []({anonymous}::std::size_t)’ may not be declared within a namespace
   __attribute__((__externally_visible__));
                                         ^
/usr/include/c++/5.2.0/new:116:41: error: ‘void {anonymous}::operator delete(void* ’ may not be declared within a namespace
   __attribute__((__externally_visible__));
                                         ^
/usr/include/c++/5.2.0/new:118:41: error: ‘void {anonymous}::operator delete [](void*)’ may not be declared within a namespace
   __attribute__((__externally_visible__));
                                         ^
/usr/include/c++/5.2.0/new:120:41: error: ‘void* {anonymous}::operator new({anonymous}::std::size_t, const {anonymous}::std::nothrow_t&)’ may not be declared within a namespace
   __attribute__((__externally_visible__));
                                         ^
/usr/include/c++/5.2.0/new:122:41: error: ‘void* {anonymous}::operator new []({anonymous}::std::size_t, const {anonymous}::std::nothrow_t&)’ may not be declared within a namespace
   __attribute__((__externally_visible__));
                                         ^
/usr/include/c++/5.2.0/new:124:41: error: ‘void {anonymous}::operator delete(void*, const {anonymous}::std::nothrow_t&)’ may not be declared within a namespace
   __attribute__((__externally_visible__));
                                         ^
/usr/include/c++/5.2.0/new:126:41: error: ‘void {anonymous}::operator delete [](void*, const {anonymous}::std::nothrow_t&)’ may not be declared within a namespace
   __attribute__((__externally_visible__));
                                         ^
In file included from /usr/include/c++/5.2.0/bits/stl_algobase.h:59:0,
                 from /usr/include/c++/5.2.0/list:60,
                 from uh-oh.cc:3:
/usr/include/c++/5.2.0/new:129:51: error: ‘void* {anonymous}::operator new({anonymous}::std::size_t, void*)’ may not be declared within a namespace
 inline void* operator new(std::size_t, void* __p) _GLIBCXX_USE_NOEXCEPT
                                                   ^
/usr/include/c++/5.2.0/new:131:53: error: ‘void* {anonymous}::operator new []({anonymous}::std::size_t, void*)’ may not be declared within a namespace
 inline void* operator new[](std::size_t, void* __p) _GLIBCXX_USE_NOEXCEPT
                                                     ^
/usr/include/c++/5.2.0/new:135:45: error: ‘void {anonymous}::operator delete(void*, void*)’ may not be declared within a namespace
 inline void operator delete  (void*, void*) _GLIBCXX_USE_NOEXCEPT { }
                                             ^
/usr/include/c++/5.2.0/new:136:45: error: ‘void {anonymous}::operator delete [](void*, void*)’ may not be declared within a namespace
 inline void operator delete[](void*, void*) _GLIBCXX_USE_NOEXCEPT { }
                                             ^
uh-oh.cc: In function ‘int main(int, char**)’:
uh-oh.cc:8:3: error: reference to ‘std’ is ambiguous
   std::list<int> l = {1,2,3,4};
   ^
<built-in>: note: candidates are: namespace std { }
In file included from /usr/include/c++/5.2.0/bits/stl_algobase.h:59:0,
                 from /usr/include/c++/5.2.0/list:60,
                 from uh-oh.cc:3:
/usr/include/c++/5.2.0/x86_64-unknown-linux-gnu/bits/c++config.h:195:1: note:                 namespace <unnamed>::std { }
 {
 ^
uh-oh.cc:8:13: error: expected primary-expression before ‘int’
   std::list<int> l = {1,2,3,4};
         ^

Interesting. It seems to detect that including a named namespace in an unnamed namespace was not the best idea. Of course, this is not what I originally intended when I encountered this error for the first time—I was rather stupidly including something that in turn included an STL header.

It is somewhat amazing to me that the simple addition of an anonymous namespace results in such an amount of error messages. The folks at StackExchange even had a contest for this sort of thing. Against their submissions, this problem very much pales in comparison.

Happy programming—be smarter than me.

Change Emacs shell's CWD with helm projectile

xenodium.com @alvaro — Thu, 08 Oct 2015 03:00:00 GMT

If using Emacs shell and helm projectile, you can wire these up to quickly change your current working directory.

(require 'helm-projectile)

(defun ar/shell-cd (dir-path)
"Like shell-pop--cd-to-cwd-shell, but without recentering."
  (unless (string-equal mode-name "Shell")
    (error "Not in Shell mode"))
  (message mode-name)
  (goto-char (point-max))
  (comint-kill-input)
  (insert (concat "cd " (shell-quote-argument dir-path)))
  (let ((comint-process-echoes t))
    (comint-send-input)))

(defun ar/helm-projectile-shell-cd ()
  "Change shell current working directory using helm projectile."
  (interactive)
  (unless (string-equal mode-name "Shell")
    (error "Not in Shell mode"))
  (let ((helm-dir-source (copy-tree  helm-source-projectile-directories-list)))
    (add-to-list 'helm-dir-source '(action . ar/shell-cd))
    (add-to-list 'helm-dir-source '(keymap . nil))
    (add-to-list 'helm-dir-source '(header-line . "cd to directory..."))
    (helm :sources helm-dir-source
          :buffer "*helm-dirs*"
          :candidate-number-limit 10000)))

Sculpin for Github pages

personal code attic — Thu, 08 Oct 2015 03:00:00 GMT

Repository setup that I've used in the past for static blog publishing via Sculpin. Model inspired by Octopress in it's repository setup.

The role of a Quality Engineer

Posts by Petr Stribny — Thu, 08 Oct 2015 00:03:39 GMT

HEAVY BULLETS (#blogtober 7)

infrequent oscillations — Wed, 07 Oct 2015 23:20:15 GMT

HEAVY BULLETS (or “superneonshootybang” as I’m now going to call it) is kinda DOOMy but with a twist - you have a revolver, 6 bullets, and the only reloads you get are when you retrieve the ones you’ve shot. There’s some plot about something going horribly wrong, yada yada. Amusingly, the graffiti seems to be from a random selection of anti-corporate & anti-rich slogans. All the neon lushness makes me happy.

Thermostat reset on Bosch WKD28350GB

xenodium.com @alvaro — Wed, 07 Oct 2015 03:00:00 GMT

My Bosch washer/dryer (WKD28350GB) stopped drying recently. Resetting the dryer's thermostat red breaker did the trick.

Edit: Similar post here.

Links - October 7th, 2015

Avy Faingezicht — Wed, 07 Oct 2015 03:00:00 GMT

SAN Intro

smcleod.net — Wed, 07 Oct 2015 03:00:00 GMT

Where’s the Fun in Skateboarding?

huphtur — Wed, 07 Oct 2015 03:00:00 GMT

Yesterday I enjoyed a great lunch with Parra, one of the guys behind Tired Skateboards. While reminiscing about our time skating together, he noted that kids nowadays don’t really understand “skateboard humor” anymore.

The comments on this tongue-in-cheek article about Nyjah Huston pretty much prove Parra’s theory. Fair to say we old guys miss Big Brother Magazine?

Point Perfect (#blogtober 6)

infrequent oscillations — Wed, 07 Oct 2015 00:56:06 GMT

Ever wanted to fly a mouse cursor into space? Defeat hordes of alien creatures in a series of mini-games? Can you drag-n-drop the alien scum into the bin of destiny? Then Point Perfect is for you. You get three rounds of silliness - lowest score ignored - and they all revolve around manipulating your mouse cursor spaceship. You have no weapons - obvs, you’re flying a mouse cursor - and the only way to eliminate aliens is to highlight one (or more if you’re nimble) with your targeting skills and let the orbital laser take them out for you.

How to extract images from a Word Document

SWilliams.io — Tue, 06 Oct 2015 04:01:01 GMT

After Googling this question and being disappointed at how much work was needed to find a good answer, the only useful post was this old post^[1] for Word 2007, here is a definitive blog post about the topic. This post assumes use of a Microsoft Word 2013 though the method would work on almost any word processor.

The by far simplest method is to save the document as a webpage. This is done by choosing the correct file type when saving as a new file. This is done as normal when saving the file but choosing a different "save as type". Below is a screenshot of saving a document called "Example".

The save screen when saving the document. Click to view full size.

Next, simply navigate to the folder where you saved the document as a webpage and you should find a new folder called "Example_files" alongside the new .htm file. Inside the folder will be every photo within the document in a .jpg format along with some meta data and other stuff you dont need to worry about.

Hooray!

Footnotes

Link to older blog post on this subject. It goes into other methods including opening the .docx as a .zip and manually saving each image. However, these are much less time efficient.

The Organ Trail (blogtober #5)

infrequent oscillations — Mon, 05 Oct 2015 19:55:33 GMT

No prizes for guessing what game this is parodying… And the intro is pitch perfect. Yes, he’s managed to break his arm driving a car down the road. Damn trails. Once you’re passed the intro (and gunfighting with HORRIFYINGLY BAD CONTROLS), you get to visit some cities… And yes, you have to contend with zombies stupid pop culture green monsters who bite the living. Apart from that, it’s your bog-standard Oregon Trail-alike (with HORRIFYINGLY BAD SHOOTING CONTROLS.

The Martian

🦉 olshansky 🦁 — Mon, 05 Oct 2015 19:11:51 GMT

Olshansky's review of The Martian

Javascript fetch node sample

xenodium.com @alvaro — Mon, 05 Oct 2015 03:00:00 GMT

Playing with node and fetch:

// Requisite: npm install node-fetch --save
// Save to fetch-demo.js
// Run: node fetch-demo.js

var fetch = require('node-fetch');

fetch("http://xenodium.com/data/javascript-fetch-node-sample/message.json\n", {
  method: 'GET',
  timeout: 5000
}).then(function(response) {
  return response.json();
}).then(function(response) {
  console.log('subject: ' + response.subject);
  console.log('body: ' + response.body);
}).catch(function(reason) {
  console.log(reason);
});

Links - October 5th, 2015

Avy Faingezicht — Mon, 05 Oct 2015 03:00:00 GMT

Delighting In The Bride

trv.rs — Mon, 05 Oct 2015 03:00:00 GMT

As for the saints in the land, they are the excellent ones, in whom is all my delight.

Psalm 16:3

Jesus, teach me to delight in the company and love of the brothers and sisters I have in Your grace. You were killed by sinners, but it was You who executed Your plan from before time to make a new people for Yourself, bringing all nations and tongues under a single banner, the cross itself, once a symbol of torture and Roman rule, now and forever an emblem of love and Your dominion over death itself!

Lord forgive me for my pride and arrogance. I have treated members of Your precious bride as unlovely, when indeed it was I who have been unloving. I have quibbled over words, looked down upon past behaviors, sneered at sin. Forgive me Lord. The family of the church is beautiful and excellent because of Your grace… and I will learn to celebrate Your goodness in every blood-washed soul. Train my eyes to see the world as You see it… every man and woman in the scope of eternity and your matchless grace covering all their life in Your perfect righteousness.

You look upon us with love, not because of who we are and what we've done, but because of who You are and what You've done. I too must learn to see this truth in those around me. Remove the logs from my eyes, keep me from being speck-centered in my relationships. Help me to love unconditionally, so that the world will see and know that I am a Your disciple. Above all else, may I see every conversation and interaction as a service opportunity, to glorify and serve You by serving those You love and died for. Thank You for this challenge and truth. I love You Lord. I pray this in Your life-giving name, amen.

Race The Sun (#blogtober 4)

infrequent oscillations — Sun, 04 Oct 2015 18:55:46 GMT

Did you play 3D Deathchase as a child? But now you have a hankering after a thoroughly modern version to replicate that nauseating motion sickness? Welcome to [Race The Sun]. Ok, that’s the title screen and looks nothing like 3D Deathchase. Once you get going, however, … You have to avoid the things and pick up the other things and there’s moving things and jumpy things and you just keep going until you splatter your intestines over something or the sun goes down.

Processing webhook events using Wishbone Part 1: Accept and validate

Jelle Smet — Sun, 04 Oct 2015 16:00:00 GMT

Many service providers offer webhooks as a means to integrate their service into another system. Webhooks are basically a form of callbacks which can be used to trigger functionality somewhere else. In the case of webhooks this is usually done by submitting some sort structured data like JSON into a …

Forest Cloak

june.kim — Sun, 04 Oct 2015 03:00:00 GMT

Geneforge (#blogtober 3)

infrequent oscillations — Sun, 04 Oct 2015 00:21:40 GMT

After reading the excellent blog of Jeff Vogel for many years, I eventually got around to buying The Geneforge Saga from GOG. It’s a charmingly old-worlde rustic RPG from the days when graphics cards cost as much as a small van and CPUs were the size of a small van. Fancy splash screen and astonishingly green menu screen for your edification. Before you start, you have to do the obligatory “assign points to categories” character creation screen.

Changing File Creation Dates in OSX

bastibe.de — Sat, 03 Oct 2015 03:00:00 GMT

On my last vacation, I have taken a bunch of pictures, and a bunch of video. The problem is, I hadn't used the video camera in a long time, and it believed that all it's videos were taken on the first of January 2012. So in order for the pictures to show up correctly in my picture library, I wanted to correct that.

For images, this is relatively easy: Most picture libraries support some kind of bulk date changes, and there are a bunch of command line utilities that can do it, too. But none of these tools work for video (exiftool claims be able to do that, but I couldn't get it to work).

So instead, I went about to change the file creation date of the actual video files. And it turns out, this is surprisingly hard! The thing is, most Unix systems (a Mac is technically a Unix system) don't even know the concept of a file creation date. Thus, most Unix utilities, including most programming languages, don't know how to deal with that, either.

If you have XCode installed, this will come with SetFile, a command line utility that can change file creation dates. Note that SetFile can change either the file creation date, or the file modification date, but not both at the same time, as any normal Unix utility would. Also note that SetFile expects dates in American notation, which is about as nonsensical as date formats come.

Anyway, here's a small Python script that changes the file creation date (but not the time) of a bunch of video files:

import os.path
import os
import datetime
# I want to change the dates on the files GOPR0246.MP4-GOPR0264.MP4
for index in range(426, 465):
    filename = 'GOPR0{}.MP4'.format(index)
    # extract old date:
    date = datetime.datetime.fromtimestamp(os.path.getctime(filename))
    # create a new date with the same time, but on 2015-08-22
    new_date = datetime.datetime(2015,  8, 22, date.hour, date.minute, date.second)
    # set the file creation date with the "-d" switch, which presumably stands for "dodification"
    os.system('SetFile -d "{}" {}'.format(new_date.strftime('%m/%d/%Y %H:%M:%S'), filename))
    # set the file modification date with the "-m" switch
    os.system('SetFile -m "{}" {}'.format(new_date.strftime('%m/%d/%Y %H:%M:%S'), filename))

Proprietary Software is an Unsafe Building Material

Dan Stroot — Sat, 03 Oct 2015 03:00:00 GMT

At one point in time I worked for an organization that banned the use of open source software. They were concerned that in the event of issues "there would be no support". At the time they believed that commercial software from companies like IBM, Oracle, Microsoft and SAP was "safer" because there was a commercial entity that stood behind the products they sold and provided support. Now that's changing...

This post Proprietary Software is an Unsafe Building Material first appeared on Dan Stroot's Blog

Seeding a sharing-economy or platform company

CRAIG KERSTIENS — Fri, 02 Oct 2015 23:55:56 GMT

These days if you’re creating a company you likely hope to accomplish more with less people, two ways of doing this fall to: The sharing economy and creating a platform. It’s easy to see the case for this when you have such unicorns like AirBnB or Uber. The opportunity for each of those to compete against hotel chains or taxi services which each need to manage their own inventory is incredibly exciting and revolutionary. In a similar fashion platforms can offer much the same, Heroku’s platform and marketplace made it easier than ever for developers to click a button and get everything they needed years ago. It’s not just their code, it’s everything from Postgres to Mongo to Logging. Or take the app store as example. Smart phones weren’t a new thing when the iPhone came out, but it was only the saviest of users that had apps installed on their windows smartphone or blackberry. The app store made the iPhone different than any other phone by allowing others to build and improve it, turning the iPhone not into a phone but a platform.

Platforms and the sharing economy both let you get further than having to take on the costs of offering the equivilent all on your own. And while a great idea to venture into one of these two areas, starting them isn’t as trivial as simply deciding to. For both of these you have issues with having a two sided market, first you have to convince the providers to come along, then the customers or vice versa. As a result of this two sided market issue the easiest way to actually start is by bootstrapping it yourself – or faking it til you make it.

What are some good examples of faking this? I’m sure you can probably find some good stories going back about AirBnB or Uber, but let’s assume times were different then. Let’s take a look at a very recent example: Lugg which just launched in the latest batch of YC. Lugg is Uber for moving essentially, allowing you to on-demand request furniture moved from one place to another. Early on Lugg built their app, then waited for requests to come in, then the founders got in a truck and moved the furniture themselves. As a customer the founders are likely providing a great experience, without ever having to tip their hat at the ways their hacking the impression of being a large well oiled machine.

But what about a platform? Slack continues to grow like wildfire as the new medium for communication. These days there’s endless integrations for slack, and I expect they’ll continue to expand what a platform for communication looks like. But a year ago they were quite a ways from having people show up at their door to add an integration. Sure there were people using them, but to expect github/trello/asana to immediately build an integration for every new flavor of the week tool would be crazy. Yet, without these integrations slack wouldn’t be nearly as useful as it is today–and probably wouldn’t have seen the growth it’s seen. In the early days of a platform the easiest way to get these integrations and partners in place is to show up and build the work yourself. Slack carried the weight early of building these integrations, much as Heroku add-ons showed up at partners offices and help write the code to get them as a provider in the marketplace. And while both Slack and Heroku are larger companies now, it still holds true for smaller ones starting today. Blockspring, a company which aims to make web services available through spreadsheets, had to do very much the same thing building their initial integrations themselves. Now with their rapidly growing user base and already large collection of APIs they may be able to shift the model, but early on that wasn’t so much an option.

If you want to build a platform, start by creating the impression of one while still carrying the load yourself. Yes, move to a true platform as soon as you can, but don’t wait for others to show up before you go that route.

The Bridge (#blogtober 2)

infrequent oscillations — Fri, 02 Oct 2015 23:43:01 GMT

Day two brings The Bridge which brings the lovely charcoal sketch aesthetic to a world of Escher-like puzzles. There’s also a Braid-alike “turn back time” mechanic but I don’t know if it becomes relevant to later puzzles or is just a “BIG WHOOPSIE UNDO” button. Our hero has fallen asleep under an apple tree. You can see where this is going… After a mini-tutorial on the controls (which also serves to wake him up), you trudge off to find his house.

HTML5 Mobile Apps, Part 5: Ecosystem Opinions

Thomas Hunter II — Fri, 02 Oct 2015 03:00:00 GMT

Grounding, part 2: For humans it means understanding

D13V — Fri, 02 Oct 2015 03:00:00 GMT

In the previous part I talked about how I encountered grounding for the first time, as it relates to a robot, connecting the abstract to the specific. Here, I’ll look more into how it relates to humans, and how it helps with understanding.

Grounding, the act of connecting the abstract to the specific, isn’t a novel technique about how to transmit information, like for example encoding morse-code in blinking. Even if we talked to each other by blinking dots and dashes, instead of pronouncing sounds with our mouths, we would still attempt to convey different ideas, symbols and concepts.

Grounding, actually, is a step in the communication process which is about transferring not just information, but also understanding. To ground your message means to encode it in terms that the other party understands and can relate to. To make these concepts understandable we need to ground them in the others’ previous knowledge of the world, by using their own language, and giving examples that they can understand and relate to.

Grounding is about the semantics of the communication, and not about the mechanics. When the person who hears us, or reads our words, or perceives our art, when they not only register our words in their ears, but also understands the essence and meaning of what we are trying to say, only then we’ve actually reached them effectively, and made them understand the point we were trying to convey.

After college, I joined the software industry and started getting a pretty decent salary. I was suddenly in a deep sea of complexity, bombarded by acronyms such as OPT, PTO, HSA, H1B, 401k, IRS, USCIS and other “fun” things. Previously, I was living in a somewhat blissful ignorance of the real-life financial problems and decisions by living in a constrained and organized academic environment, but as a newly employed I had to take care of a lot of those nuisances.

At some point I needed to understand what is 401k. I had heard that it is about retirement savings, but didn’t have much motivation and incentive to learn about it and to contribute. It seemed like something I don’t need to worry about yet, and I could take care of it later. The “helping” materials from my first job targeted people who understand the concept, probably through a previous job, and didn’t include explanations of the simple definitions, leaving out the “why”’s. All the information about it was reaching me but I did not understand what it meant. The materials directly jumped to recommendations and strategies, and I found myself utterly confused and not sure where to start from. Without grokking the basics, I felt helpless with the more advanced terminology.

Unable to connect any of the abstract terms from the materials to my practical reality, I looked around for other explanations and started watching Khan Academy videos on finance. Each video lecture only took about 10 minutes, and contained a specific imaginary example of a person adding money to their 401k over time, and considering possible withdrawal options, based on specific situations. Example situations included emergencies, switching between different types of IRAs, investment outcomes and discussed the differences between the types of retirement investing account - traditional, Roth and 401k. The specific examples in each lecture made the terms much more clear, because it built this imaginary world, similar to the one I live in. Furthermore, the examples avoided legal jargon, and used words I could understand. Maybe they were a little bit less precise and complete than reading the actual law, but communicated better, at least to a novice mind like mine. Watching these video lectures I gained a bunch of confidence about these saving techniques which opened the possibility for me to seek more information on the subject. The specific examples built a bridge of understanding that I could cross, starting from the world I knew, to the abstract world of definitions.

In the information channel above, the grounding is the step after having transferred the data, and involves decoding the meaning. It happens on the receiver’s side, but if we as communicators don’t present our data in a format that is easily understandable by the other side, then even if we successfully say our words and they register the sounds correctly in their ears, they will still be oblivious about the meaning that we are trying to convey. Effectively, all our speech would be equivalent to a bumbling noise in their ears. On the other hand, if we tailor our message to them, we can help with their understanding.

Thats also why grounding depends on who we are trying to communicate to. There isn’t a single way to explain a concept which would make it understood by everyone. To communicate effectively we have to use explanation and examples that both sides know and understand, and that really depends on who other side is. Formulas and definitions apt to explain basic physics laws to graduate students would not be clear to preschoolers, and vise versa. The preschoolers wouldn’t understand the differential equations, while the graduate students wouldn’t find the hand-wavy simplistic explanations clear enough to be practical.

For the geeks of us, part of the theory of why grounding works this way is due to Snannon’s noisy channel theorem, which says that the amount of communication in the channel in the best case is proportional to the mutual information between the two sides. Using that mutual information would make the other side understand better the message.

An interesting corollary is to avoid adding any information that they don’t know. Using fancy words when speaking to a person who doesn’t know them would not help them understand the meaning better. Instead, it will only stroke the speaker’s ego about how smart and knowledgeable they are to use these words. Any information that is unknown to the other side will end up wasted as noise, until they can understand it.

In the next parts, I will dive in into more examples of using grounding in tricky situations.

#blogtober

infrequent oscillations — Thu, 01 Oct 2015 23:50:42 GMT

Or “let’s try #septemblog again”. For each day in #blogtober, I’m supposed to be playing one of my many unplayed games that I’ve accumulated through things like Humble Bundle. First up is Dungeon Souls - a “2-D top-down hack n’ slash/RPG game”. (Not sure why FRAPS isn’t capturing the full glory of the title screen there.) Which is exactly what it says on the tin - you pick your bog-standard class of RPG adventurer,

Snail organs, immunity, and ageing

Liza Shulyayeva — Thu, 01 Oct 2015 15:54:41 GMT

Organs! Immune systems! Old age! Dying from old age!

I’m super excited. I was thinking: “How do I implement old age and death, and health in general? Is health just energy? When the snail is out of energy, it dies? But that seems too simplistic.”

Extract dominant colors in images

xenodium.com @alvaro — Thu, 01 Oct 2015 03:00:00 GMT

There's a handy HN post pointing to Javier López's Using imagemagick, awk and kmeans to find dominant colors in images. A comment also highlights color-extract, written in Go.

Links - October 1st, 2015

Avy Faingezicht — Thu, 01 Oct 2015 03:00:00 GMT

HTML5 Mobile Apps, Part 4: Live Operations

Thomas Hunter II — Thu, 01 Oct 2015 03:00:00 GMT

Anvil: time for a change

zserge's blog — Thu, 01 Oct 2015 03:00:00 GMT

Those of you who regularly check my blog may remember that I was going to redesign Anvil from scratch. Unfortunately, it was an unavoidable step. We had performance issues and it turned out that blindly copying modern web metaphors to Android doesn’t work well. I am sorry, I had to change a lot of Anvil APIs since then. Those of you who have their apps written with Anvil 0.0.12 most likely would prefer to stick with the older version.

[Computing] The Straight Line Principle

devever.net/~hl — Thu, 01 Oct 2015 03:00:00 GMT

Which of the following psuedocode examples do you prefer?

Links - September 30th, 2015

Avy Faingezicht — Wed, 30 Sep 2015 03:00:00 GMT

Today I discovered an amazing LinkedIn feature that I did not know existed. It is called "Connections In The News". With the subject "News about Aaron Dallek" sitting in my inbox, LinkedIn's email really intrigued me. Clicking through took me to a New York Times article about Opternative, the startup where I used to work, and where Aaron is CEO. Quite simply, LinkedIn parsed Aaron's name from the piece, and notified me that one of my contacts was mentioned in it. How the software distinguishes between various people named John Smith, or whether it tries at all, is unclear to me, but the feature provided me with a pleasant surprise.

HTML5 Mobile Apps, Part 3: Distribution

Thomas Hunter II — Wed, 30 Sep 2015 03:00:00 GMT

Authentication Quickstart with Express, Passport and Sequelize

Posts by Petr Stribny — Tue, 29 Sep 2015 21:39:57 GMT

Links - September 29th, 2015

Avy Faingezicht — Tue, 29 Sep 2015 03:00:00 GMT

HTML5 Mobile Apps, Part 2: Performance

Thomas Hunter II — Tue, 29 Sep 2015 03:00:00 GMT

Grounding, part 1: An intro from a robot

D13V — Tue, 29 Sep 2015 03:00:00 GMT

This is the first part of several essays on the principle of grounding. I feel that this principle helps me communicate and think better. I’ll try to keep each essay short

A few years ago I was doing my masters, with the goal of using language to command robots. I was very excited because the research was very interesting to me - I needed to understand how to write software for robots, how the robot works as a system, and various fascinating aspects of human-robot interaction and linguistics. It felt super futuristic to be at the meeting point of so many disciplines.

Specifically, I was trying to teach a robot to follow some simple commands such as Pick up the cup. I was working with a robot which had two hands, and a 3D scanning sensor for eyes, and was usually facing a table with a bunch of standard household objects on top of it. If the robot was facing a specific table with specific object configuration, it is possible to program and hard-code the robot arm to move towards the cup, grab the cup, and then lift it. However, as soon as something tiny changes in the configuration, the position of the cup the robot, or the command, then the hard-coded solution would no longer work, instead making the robot wave its hands in air clumsily. Programming the robot in a single situation may be fun, but doing the same thing over and over will get boring and annoying quite fast, and on top of it wouldn’t be effective.

My task was to make the robot succeed when the cup looks differently, or is at a different location, or even if the command asked for a different item. In each new environment, the robot needed to connect the abstract symbol for cup, bowl, bottle from the command to the sensory information from the environment, figure out the right motions and execute them.

Luckily, I didn’t have to start from scratch. I stepped on the shoulders of giants, my professor and the senior graduate students, who had already invented algorithms for this meeting point of disciplines. They created techniques for connecting the symbolic command to the physical reality, albeit for a very different robot, in a different setting. My task was to port the algorithm to this new robot and make sure it works, helping justify the algorithm’s ability to adapt to new situations.

The algorithm connected the text of the command and the sensory perceptions of the robot and its imaginary motions. This is grounding, as I came to know it. Grounding is the connecting between something completely abstract and symbolic, such as the symbols for the action pick up and the object cup and on the other hand the specific sets of 3D pixels and specific hand trajectories and motor settings. The latter ones exist in the reality, they are very exact - they are solid ground. The former are just the names of items or motions and could resolve differently depending on the situation.

In this project, grounding is the same as name resolution. Finding the best object which corresponds to the meaning of the cup, given the whole context of the command gets to be named the cup. Finding the best trajectory of the robot hand that corresponds to the action of pick up, again gets to be named pick up.

This connection, the grounding, bridged the gap between the abstract, clean and imaginary part of the robot’s understanding, the name of each object and trajectory, and the concrete, noisy and complicated physical world. Without crossing this divide somehow, the robot can’t to do anything of physical importance correctly. These physical actions are what made the robot a ROBOT, instead of just a normal computer. The algorithm built a bridge for the robot between the abstract and the concrete, so that the robot can cross it every time it needs to do something of physical importance.

The specific algorithm may have been clever, but what I found to be the really wise part, was the step of connecting the abstract to the specific. Every time I was trying explain something and provided specific examples of my abstract ideas I noticed that not only I was able to communicate these ideas more clearly, I also understood my ideas better. Grounding became a tool for better communicating and a tool for clearer thinking.

The next several essays will dive more deeply into the various aspects and applications of this tool.

Numpy Broadcasting Rules

bastibe.de — Tue, 29 Sep 2015 03:00:00 GMT

They say that all arithmetic operations in Numpy behave like their element-wise cousins in Matlab. This is wrong, and seriously tripped me up last week.

In particular, this is what happens when you multiply an array with a matrix[1] in Numpy:

     [[  1],           [[1, 2, 3],       [[ 1,    2,   3],
      [ 10],       *    [4, 5, 6],   =    [ 40,  50,  60],
      [100]]            [7, 8, 9]]        [700, 800, 900]]

 [  1,  10, 100]       [[1, 2, 3],       [[  1,  20, 300],
        OR         *    [4, 5, 6],   =    [  4,  50, 600],
[[  1,  10, 100]]       [7, 8, 9]]        [  7,  80, 900]]

They behave as if each row was evaluated separately, and singular dimensions are repeated where necessary. It helps to think about them as row-wise, instead of element-wise. This is particularly important in the second example, where the whole 1d-array is multiplied with every row of the 2d-array.

Note that this is not equivalent to multiplying every element as in [a[n]*b[n] for n in range(len(a))]. I guess that's why this is called broadcasting, and not element-wise.

[1] "matrix" here refers to a 2-d numpy.array. There is also a numpy.matrix, where multiplication is matrix multiplication, but this is not what I'm talking about.

Benchmark of Python JSON libraries

Artem Krylysov — Tue, 29 Sep 2015 03:00:00 GMT

A couple of weeks ago after spending some time with Python profiler, I discovered that Python’s json module is not as fast as I expected. I decided to benchmark alternative JSON libraries.

Libraries #

python-cjson, yajl-py and jsonlib are not included in the benchmark, they are not in active development and don’t support Python 3.

simplejson and ujson may be used as a drop-in replacement for the standard json module, but ujson doesn’t support advanced features like hooks, custom encoders and decoders.

You can change your imports this way to use an alternative library:

import ujson as json

Interpreters #

Python (CPython) 2.7.12
Python (CPython) 3.5.2
PyPy 5.3.0

Methodology #

The tests were performed on MacBook Pro Late 2013 (2.6 GHz Intel Core i5, 8 GB 1600 MHz DDR3, Mac OS 10.11.6). Every test runs 100 times.

Test data
File name	File size	Description
twitter.json	632 KB	Single large JSON (source)
one-json-per-line.jsons.txt	176 KB	Collection of 1000 JSON objects (source)

I published the source code of the benchmark on GitHub. You can clone it and rerun if you want to check it by yourself or if a new version of an alternative JSON library is released.

Results #

Python 2.7
	json	simplejson	ujson
loads (large obj)	1.140	0.441	0.448
dumps (large obj)	0.564	0.630	0.459
loads (small objs)	1.190	0.579	0.195
dumps (small objs)	0.910	1.641	0.304

Python 3.5
	json	simplejson	ujson	rapidjson
loads (large obj)	0.600	0.698	0.605	0.634
dumps (large obj)	0.673	0.629	0.381	0.365
loads (small objs)	0.801	1.091	0.322	0.531
dumps (small objs)	1.213	2.038	0.285	0.234

PyPy 5.3
	json	simplejson
loads (large obj)	0.545	1.876
dumps (large obj)	0.632	3.974
loads (small objs)	0.271	1.651
dumps (small objs)	0.719	2.404

Conclusion #

The numbers speak for themselves. If your application is dealing with a big amount of JSON data and doesn't use any advanced features of built-in json module, you should probably consider switching to ujson.

One Weird Trick to Write Better Code

Evan Todd — Tue, 29 Sep 2015 00:00:00 GMT

Developers hate him!

We'll cover some standard tips and tricks here, but we're not really interested in those. We're looking for the One Weird Trick to rule them all. Hopefully each trick we encounter brings us closer to coding Mecca.

In the beginning

The first video game I ever wrote was called Ninja Wars.

Yes, that is an HTML table of images. I changed the src attribute to move stuff around.

Find a word with regex and WordNet

xenodium.com @alvaro — Mon, 28 Sep 2015 03:00:00 GMT

Recently wanted to come up with a random keyword. Querying WordNet and a regular expression did the job.

Installed WordNet on Mac:

$ brew install wordnet

Want a word ending in "esome"?

$ wn esome -grepn -grepv -grepa -grepr | egrep -o -e "\w*esome\b" | sort | uniq

adventuresome
awesome
blithesome
bunglesome
cuddlesome
esome
fivesome
gruesome
lithesome
lonesome
lovesome
meddlesome
mettlesome
nettlesome
threesome
tiresome
torturesome
troublesome
unwholesome
venturesome
wholesome

Break hard and early

june.kim — Mon, 28 Sep 2015 03:00:00 GMT

Links - September 28th, 2015

Avy Faingezicht — Mon, 28 Sep 2015 03:00:00 GMT

My apoligies for the lack of links on Friday. Hopefully today's set makes up for that.

2D Contouring

Matt Keeter — Mon, 28 Sep 2015 03:00:00 GMT

An overview of various algorithms

HTML5 Mobile Apps, Part 1: Development and Debugging

Thomas Hunter II — Mon, 28 Sep 2015 03:00:00 GMT

Python Numeric Performance

bastibe.de — Mon, 28 Sep 2015 03:00:00 GMT

Recently, I was working on a dynamic programming algorithm that involves a lot of number crunching in nested loops. The algorithm looks like this:

def y_change_probability_python(oct_per_sec):
    """ ... """
    b = 1.781
    mu = -0.301
    return 1/(2*b)*math.exp(-abs(oct_per_sec-mu)/b)

def y_idx_range_python(y_idx, max_y_factor, height):
    """ ... """
    y = (y_idx/height)*(max_y-min_y)+min_y
    y_lo = max(y/max_y_factor, min_y)
    y_hi = min(y*max_y_factor, max_y)
    y_lo_idx = int((y_lo-min_y)/(max_y-min_y)*height)
    y_hi_idx = int((y_hi-min_y)/(max_y-min_y)*height)
    return y_lo_idx, y_hi_idx

def find_tracks_python(cost_matrix, delta_x):
    """ ... """
    tracks = np.zeros(correlogram.shape, dtype=np.int64)
    cum_cost = np.zeros(correlogram.shape)

    max_y_factor = (2**5)**(delta_x)
    height = correlogram.shape[1]

    probabilities = np.empty((height, height))
    for y_idx in range(height):
        y = (y_idx/height)*(max_y-min_y)+min_y
        for y_pre_idx in range(*y_idx_range_numba(y_idx, max_y_factor, height)):
            y_pre = (y_pre_idx/height)*(max_y-min_y)+min_y
            doubles_per_x = math.log2((y/y_pre)**(1/delta_x))
            probabilities[y_idx, y_pre_idx] = y_change_probability_numba(doubles_per_x)

    for x_idx, cost_column in enumerate(cost_matrix):
        if x_idx == 0:
            cum_cost[x_idx] = cost_column
            continue
        for y_idx, cost in enumerate(cost_column):
            for y_pre_idx in range(*y_idx_range_numba(y_idx, max_y_factor, height)):
                weighted_cum_cost = cum_cost[x_idx-1, y_pre_idx] + cost*probabilities[y_idx, y_pre_idx]
                if weighted_cum_cost > cum_cost[x_idx, y_idx]:
                    cum_cost[x_idx, y_idx] = weighted_cum_cost
                    tracks[x_idx, y_idx] = y_pre_idx
            cum_cost[x_idx, y_idx] = cum_cost[x_idx-1, tracks[x_idx, y_idx]] + cost

    return tracks, cum_cost

I'm not going into the details of what this algorithm does, but note that it iterates over every column and row of the matrix cost_matrix, and then iterates over another range previous_y_range for each of the values in cost_matrix. On the way, it does a lot of basic arithmetic and some algebra.

The problem is, this is very slow. For a $90 \times 200$ cost_matrix, this takes about 260 ms. Lots of loops? Lots of simple mathematics? Slow? That sounds like a perfect match for Numpy!

If you can express your code in terms of linear algebra, Numpy will execute them in highly-optimized C code. The problem is, translating loops into linear algebra is not always easy. In this case, it took some effort:

def y_change_probability_numpy(doubles_per_x):
    """ ... """
    b = 1.781
    mu = -0.301
    return 1/(2*b)*np.exp(-np.abs(doubles_per_x-mu)/b)

def find_frequency_tracks_numpy(cost_matrix, delta_x):
    """ ... """
    tracks = np.zeros(cost_matrix.shape, dtype=np.int)
    cum_cost = np.zeros(cost_matrix.shape)

    max_y_factor = (2**5)**(delta_t) # allow at most 5 octaves per second (3 sigma)
    height = cost_matrix.shape[1]

    # pre-allocate probabilities matrix as minus infinity. This matrix
    # will be sparsely filled with positive probability values, and
    # empty values will have minus infinite probability.
    probabilities = -np.ones((height, height))*np.inf
    for y_idx in range(probabilities.shape[0]):
        y = (y_idx/height)*(max_y-min_y)+min_y
        y_pre_idx = np.arange(int((max(y/max_y_factor, min_y)-min_y)/(max_y-min_y)*height),
                              int((min(y*max_y_factor, max_y)-min_y)/(max_y-min_y)*height))
        y_pre = (y_pre_idx/height)*(max_y-min_y)+min_y
        doubles_per_x = np.log2((y/y_pre)**(1/delta_x))
        probabilities[y_idx, y_pre_idx] = y_change_probability(doubles_per_x)

    cum_cost[0] = cost_matrix[0]
    for x_idx in range(1, len(cost_matrix)):
        cost_column = cost_matrix[x_idx:x_idx+1] # extract cost_column as 2d-vector!
        weighted_cum_cost = cum_cost[x_idx-1] + cost_column.T*probabilities
        tracks[x_idx] = np.argmax(weighted_cum_cost, axis=1)
        cum_cost[x_idx] = cum_cost[x_idx-1, tracks[x_idx]] + cost_column

    return tracks, cum_corrs

This code does not look much like the original, but calculates exactly the same thing. This takes about 15 ms for a $90 \times 200$ cost_matrix, which is about 17 times faster than the original code! Yay Numpy! And furthermore, this code is arguably more readable than the original, since it is written at a higher level of abstraction.

Another avenue for performance optimization is Numba. Numba applies dark and powerful magic to compile humble Python functions into blazingly fast machine code. It is proper magic, if you ask me. Simply add an innocuous little decorator to your functions, and let Numba do it's thing. If all goes well, your code will work just as before, except with unheard-of performance:

@jit(numba.float64(numba.float64), nopython=True)
def y_change_probability_numba(doubles_per_x):
    ...

@jit((numba.int64, numba.float64, numba.int64), nopython=True)
def y_idx_range_numba(y_idx, max_y_factor, height):
    ...

@jit((numba.float64[:,:], numba.float64), nopython=True)
def find_tracks_numba(cost_matrix, delta_t):
    ...

However, Numba is no silver bullet, and does not support all of Python yet. In the present case, it is missing support for enumerate for Numpy matrices. Thus, I had to rewrite the first two loops like this:

# python version
for x_idx, cost_column in enumerate(cost_matrix):
    ...

# numba version
for x_idx in range(len(cost_matrix)):
    cost_column = cost_matrix[x_idx]
    ...

Another area that proved problematic is N-D slice writing. Instead of using expressions like m1[x,y:y+3] = m2, you have to write for idx in range(3): m1[x,y+idx] = m2[idx]. Not a difficult transformation, but it basically forced me to unroll all the nice vectorized code of the Numpy version back to their original pure-Python form. That said, Numba is getting better and better, and many constructs that used to be uncompilable (yield) are not a problem any more.

Anyway, with that done, the above code went down from 260 ms to 2.2 ms. This is a 120-fold increase in performance, and still seven times faster than Numpy, with minimal code changes. This is proper magic!

So why wouldn't you just always use Numba? After all, when it comes down to raw performance, Numba is the clear winner. The big difference between performance optimization using Numpy and Numba is that properly vectorizing your code for Numpy often reveals simplifications and abstractions that make it easier to reason about your code. Numpy forces you to think in terms of vectors, matrices, and linear algebra, and this often makes your code more beautiful. Numba on the other hand often requires you to make your code less beautiful to conform to it's subset of compilable Python.

Searching for swear words in the Enron corpus

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Mon, 28 Sep 2015 01:19:57 GMT

I recently downloaded the Enron corpus to try to visualize some interesting features. As a first step, I wanted to analyse how many swear words are used in the corpus. I wanted to find out if there are some users with a “saltier“ language than others.

Processing the data

I am only using the simplest procedure for walking through the data: First, I defined a list of offensive words (with a little help from Wikipedia, of course). Second, I used a basic Python script to process each e-mail, tokenize it, and check the words against the list of profanity. The counts are then tallied and printed out. Here’s a rough draft of the script:

#!/usr/bin/env python

import collections
import email
import hashlib
import nltk
import os

directory           = "maildir/"
swearWordsPerPerson = collections.Counter()
swearWordsTotal     = collections.Counter()
swearWords          = set()

parser = email.parser.HeaderParser()

with open("Swear_words.txt") as f:
  for line in f:
    swearWords.add( line.rstrip() )

for root, _, files in os.walk(directory):
  p = os.path.normpath(root)
  p = p.split(os.sep)

  person = ""
  if len(p) >= 2:
    person = p[1]

  print("Processing '%s'..." % person)

  for name in files:
    path = os.path.join(root,name)
    with open(path, errors="ignore", encoding="utf-8") as f:
      text   = f.read()
      text   = parser.parsestr(text, headersonly=True).get_payload()
      text   = text.encode("utf-8")
      words  = nltk.word_tokenize(text.decode("utf-8").lower())
      for index,word in enumerate(words):
        if word in swearWords:
          swearWordsPerPerson[person] += 1
          swearWordsTotal[word]       += 1

  print("...finished")
  print("%s: %d" % (person, swearWordsPerPerson[person]))

print("THE SWEARERS\n")

for person, numSwears in swearWordsPerPerson.most_common():
  print("  %s: %d" % (person, numSwears))

print("")
print("THE SWEARS\n")

for swear, number in swearWordsTotal.most_common():
  print("  %s: %d" % (swear, number))

The conversion of text from each e-mail looks needlessly elaborate, but it is required to ensure that words in the e-mail headers are not counted. Of course, I am not trying to detect any fancy things such as messages that have been forwarded multiple times. Furthermore, I have excluded some words from the list of swears in order not to trigger any false positives. One of the executives is called “Dick Jenkins”, and some of his co-workers refer to him by his first name, so…

The swearers

First, let’s look at a histogram of the absolute number of swears:

We can see that there are only few people who seem to use many swears. However, this is not normalized against the number of e-mails each person received. Let’s do this next!

The pattern is similar, but the outliers are now “normalized” in a sense. If we focus on a normalized swear count of larger than 0.25 (with a unit of “swears per e-mail”), we are left with the following persons:

lucci-p   0.33
lenhart-m 0.28
brawner-s 0.27
quigley-d 0.19
ring-a    0.18
dorland-c 0.15

All the remaining 144 (!) persons use less than 0.15 swear words per e-mail.

The swears

So, what are the top ten swear words in all e-mails? Without further ado, here is a partially censored list (I don’t want to lose my family-friendly rating):

hell   2033
a**:   1724
sh*t   1583
sex    1521
balls  1235
d*mn    915
b*tch   711
f*ck    690
butt    615
crap    612

Conclusion

Does this mean that the Enron people like to swear? Unfortunately, no. First of all, the context of a word is extremely important. In some cases, hell might be a misspelling of he’ll. Second, the swear count might be multiplied by people forwarding (or replying) e-mails without removing the old content.

Upon further inspection, it turns out that most of the swear words are used in personal communications. People are forwarding the usual dirty jokes among their friends and co-workers, who in turn reply with a salty joke themselves.

The take-away messages from this experiment are:

Figuring out words in context is hard.
Analysing a corpus such as the Enron e-mails is also hard.
Natural language is even harder.

Macaroons 101: Contextual Confinement

Evan Cordell — Sun, 27 Sep 2015 15:35:45 GMT

Macaroons, like Fezzes, are cool. If you find yourself disagreeing, it’s possible you’re thinking of the wrong sort of macaroons , or you’ve yet to be convinced. This is the first in a series of posts in which I intend to explain why macaroons are so interesting. We’ll start by motivating them and defining them, and later expand on the theory behind them and study examples of how they’re applicable in the real world. The name “macaroon” comes from the Google research paper that introduced them, a play on the ubiquitous browser “cookie.

2015-09-27

ho.dges.online — Sun, 27 Sep 2015 03:00:00 GMT

Hitch

🦉 olshansky 🦁 — Sat, 26 Sep 2015 06:23:32 GMT

Olshansky's review of Hitch

Soundcloud's Go best practices (GopherCon 2014)

xenodium.com @alvaro — Sat, 26 Sep 2015 03:00:00 GMT

Having watched the video, some takeaways:

Single GOPATH

$GOPATH/src/github.com/soundcloud/foo

Repo structure

github.com/soundcloud/whatever

README.md
Makefile
main.go
support.go
foo
1. foo.go
2. bar.go
whatever-server
1. main.go
wharever-worker
1. main.go

Formatting and style

Use gofmt.

Google's codereview guidelines.

Avoid named return parameters.

Avoid make and new (unless you know sizes).

Use struct{} for sentinel values: sets, signal chans.

Conveys no information in it this part.
Instead of empty interface.
instead of boolean.

Break long lines at parameters

No need to compact.
Keep trailing coma in last argument.

Flags

func main() {
  var (
    foo = flags.String("foo\n", "doch\n", "...")
    bar = flat.Int("bar\n", 34, "...")
  )
  flag.Parse()
  // ...
}

Logging

package log
Telemetry
Push model (gets expensive over time)
1. Graphite
2. Statsd
3. AirBrake
Pull model (chosen)
1. expvar
2. Prometheus

Testing

package testing
1. Unit tests
2. reflect.DeepEqual
Integration
1. Use flags for starting services
2. // +build integration

Code validation

On Save
1. Go fmt
2. Go import (go fmt++)
On Build
1. Go vet
2. Golint
3. Go test
On Deploy
1. go test -tags=integration
GoCov?

Dependency management

Unimportant projects
1. go get -d (and hope)
Important
1. VENDOR (ie. copy into your repo)
  1. Git submodules (no!).
  2. Git subtrees (seem OK).
  3. Tool (godep?).
  4. Build
  5. For binaries (use _vendor subdir)

Born Yesterday by Philip Larkin

ho.dges.online — Sat, 26 Sep 2015 03:00:00 GMT

For Sally Amis

Tightly-folded bud,
I have wished you something
None of the others would:
Not the usual stuff
About being beautiful,
Or running off a spring
Of innocence and love —
They will all wish you that,
And should it prove possible,
Well, you’re a lucky girl.

But if it shouldn’t, then
May you be ordinary;
Have, like other women,
An average of talents:
Not ugly, not good-looking,
Nothing uncustomary
To pull you off your balance,
That, unworkable itself,
Stops all the rest from working.
In fact, may you be dull —
If that is what a skilled,
Vigilant, flexible,
Unemphasised, enthralled

Is it Secure?

Dan Stroot — Sat, 26 Sep 2015 03:00:00 GMT

Most information security practitioners do a poor job explaining the relative nature of security. Secure relative to what? A bot that looks for un-patched systems? A determined hacker? A nation-state? Security is a "scale of difficulty" – like mountain climbing. Everest was thought to be unclimbable until Sir Edmund Hillary did it.

This post Is it Secure? first appeared on Dan Stroot's Blog

Why don't we just do solar instead of nuclear?

Negative Feedback — Fri, 25 Sep 2015 17:31:24 GMT

I've been seeing a lot of people complaining about the nuclear deal. There seems to be an especially vocal group who feel that we should be investing in renewables rather than nuclear. Some of these people hold up Fukushima as a warning about how bad things can go with nuclear power. There's a lot to say about this topic, and I'm not going to take the time to write a book about it, just drop some links and facts for the interested reader.

First, Fukushima is not as bad as you may think. Specifically, no-one died or is projected to die due to radiation released at Fukushima.

Second, we want less CO₂ to be produced in the production of our electricity needs, we need to move away from fossil fuels. The nuclear advocates and the solar advocates usually agree on this. The problem is that nuclear fills a niche that solar really struggles with: base load. I went hunting for the most recent information on the South African situation, and I came across this presentation which shows the problem with South African power quite clearly - there are large peaks in usage in the morning and the afternoon in winter. This Eskom infographic has a nicer display of the winter and summer behaviour. I also tried to find good information on the insolation in SA and ended up choosing a weather station in the Richtersveldt for my data since it is located in a nice open area in the best solar position in the country. When you plot these together, you get this graph:

Notice how the peaks take place at times where there isn't really much sun. This means solar power can't really supply the power for those peaks directly. More importantly, it can't help with the minimum demand at 02:00. We would have to store the energy somehow.

Another issue often bandied about in these conversations is how expensive the nuclear plants would be. The figures I've heard are that the nuclear deal is perhaps going to cost around R1.2T for 9.6 GW of nuclear nameplate capacity. People look at that and think "if Solar PV costs around $3/W, we could build the same solar capacity for a fraction of the price". The problem, of course, is that the capacity factors for the two technologies aren't the same. For nuclear they're around 90%, while for solar they're around 20 %. This means we would need to install around 43 GW of nameplate capacity solar to produce the same amount of electricity, which costs more than R1.2T just by itself at $3/W. Then we would have to pay for the storage and the refitting of our grid to allow for the distributed generation which the solar plants would inevitably represent. I love solar, but we're not at the point where it can do the base load generation we would be purchasing with nuclear.

Wild snails around the world

Liza Shulyayeva — Fri, 25 Sep 2015 01:16:16 GMT

Did you know that there is an entirely separate formula to calculate the distance between two points on the surface of a sphere? I mean…it kind of makes sense when you think about it. The shortest distance between two points without accounting for the curvature of the sphere would go right through the sphere itself, making it pretty useless when calculating physical distance on, for example, the surface of the Earth.

Group And Community Projects

Venam's Blog — Patrick Louis (Lebanon) — Fri, 25 Sep 2015 00:00:00 GMT

Hello fellow readers, In this post I'll talk about group and community projects. Everyone has been part of multiple group projects throughout their life. From school researches, to university presentations, to work. It's only by sharing ideas and finding common grounds that goals can be achieved. However, it's hard to maintain cohesion and investment in a group. There's a multitude of problems that arise, the reader can certainly assert it.

Python Code Optimization Tips

denvaar's website — Thu, 24 Sep 2015 08:40:09 GMT

Tips from Guido Van Rossum about code optimization.

Automated Testing of Dotfiles

Michael Mior — Thu, 24 Sep 2015 03:00:00 GMT

Several years ago I started managing my dotfiles based on Zach Holman’s dotfiles repo. His setup is quite nice and I found it relatively easy to adapt to my own purposes. My workflow generally consisted of making a bunch of local changes until I was happy and then pushing to my own GitHub fork.

The big problem I eventually found is that I wasn’t fully capturing the correct steps to reproduce my environment. Every time that I tried to install my dotfiles on a new machine, I would be met with several errors that I would eventually resolve. The fix would not always result in something which was reproducible on another machine. I wanted a solution that would let me automatically test that my dotfiles would cleanly install every time I pushed to GitHub, so I turned to Docker.

Traditional CI services would have been a bit of a pain to use with all the packages that needed to be installed. Docker Hub made things nice and easy. My Dockerfile simply installs the necessary OS packages, adds a new user and then tries to run my install script. I currently don’t have any other testing other than to ensure that the script exits without error, but this has already saved me a lot of trouble.

Update: I have since switched to using Travis CI as I do with my other projects. It turns out this is easier than I expected. I still haven’t explicitly added any tests but even being able to confirm that the installation steps succeed is useful to ensure nothing breaks.

Links - September 24th, 2015

Avy Faingezicht — Thu, 24 Sep 2015 03:00:00 GMT

Bash SSH host management

Thought Eddies — Wed, 23 Sep 2015 21:03:00 GMT

Bash SSH host management

Me and Earl and the Dying Girl

🦉 olshansky 🦁 — Wed, 23 Sep 2015 20:57:25 GMT

Olshansky's review of Me and Earl and the Dying Girl

Mr. Robot: Season 1

🦉 olshansky 🦁 — Wed, 23 Sep 2015 18:07:05 GMT

Olshansky's review of Mr. Robot: Season 1

Sync pip with Mac OS updates

xenodium.com @alvaro — Wed, 23 Sep 2015 03:00:00 GMT

My pip installation recently broke after a Mac OS update.

$ pip
Traceback (most recent call last):
  File "/usr/local/bin/pip\n", line 5, in <module>
    from pkg_resources import load_entry_point
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py\n", line 2793, in <module>
    working_set.require(__requires__)
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py\n", line 673, in require
    needed = self.resolve(parse_requirements(requirements))
  File "/System/Library/Frameworks/Python.framework/Versions/2.7/Extras/lib/python/pkg_resources.py\n", line 576, in resolve
    raise DistributionNotFound(req)
pkg_resources.DistributionNotFound: pip==1.1

Updating my pip installation fixes the break:

$ sudo easy_install -U pip

Links - September 23rd, 2015

Avy Faingezicht — Wed, 23 Sep 2015 03:00:00 GMT

I am amazed by the fact that Caltrain ridership tracks the Nasdaq. This is the kind of relationship that I would never think about on my own, but once I read it, it clicked and makes perfect sense. I wonder if similar correlations can be found for other cities and their public transit.

Wooden Table

june.kim — Tue, 22 Sep 2015 03:00:00 GMT

Links - September 22nd, 2015

Avy Faingezicht — Tue, 22 Sep 2015 03:00:00 GMT

Replacing Junos Pulse with OpenConnect

smcleod.net — Tue, 22 Sep 2015 03:00:00 GMT

In an attempt to avoid using the Juniper Pulse (Now Pulse Secure) VPN client we tried OpenConnect but found that DNS did not work correctly when connected to the VPN. This bug has now been resolved recently but has not made it’s way into a new build, in fact there have been no releases for 6 months.

Luckily the OpenConnect was not too difficult to build from source.

Build OpenConnect on OSX

Remove old openconnect and install deps

brew remove openconnect
brew install libxml2 lzlib openssl libtool libevent

Build openconnect

wget git.infradead.org/users/dwmw2/openconnect.git/snapshot/0f1ec30d17aa674142552e275bf3fac30d891b39.tar.gz
tar zxvf 0f1ec30d17aa674142552e275bf3fac30d891b39.tar.gz
cd openconnect-0f1ec30

LIBTOOLIZE=glibtoolize ./autogen.sh
PATH=/usr/local/opt/gettext/bin:$PATH
./configure
make
make install

To connect

sudo openconnect --juniper -u myusername www.myserver.com

If you’re comfortable with allowing admin users to run openconnect without entering a sudo password, add the following using sudo visudo:

As a Customer, I Want To See Up-to-Date Costs I've Incurred, Using a Web Application

Databases & Life — Tue, 22 Sep 2015 03:00:00 GMT

I think it's important, that in a software development company, as all others, customers get transparent up-to-the-minute information about what costs they have incurred. Alas my company can't provide that at the moment, due to our tooling (this is a deficiency, or at least difference of opinion, between me and LiquidPlanner, the tool we use for tracking time.) This is the email I sent to LiquidPlanner support explaining my position:

I and my team do work. We use LiquidPlanner to track our done time. I can export the timesheet data and write invoices to my customers once a month. So far so good.

Identifying redundant edges in a dependency graph

Tafkas Blog — Tue, 22 Sep 2015 03:00:00 GMT

An ETL import graph is build on logical dependencies of the jobs to each other. So typically a SQL transformation job depends on all the previous jobs that create the tables used in the query. But once there are a certain number of jobs, dependencies often get a bit more complicated and some of them become redundant in the process. A simple example can be seen in the dependency graph from figure, where the three red edges are redundant.

Black Mirror: Season 2

🦉 olshansky 🦁 — Mon, 21 Sep 2015 17:30:12 GMT

Olshansky's review of Black Mirror: Season 2

Black Mirror: Season 1

🦉 olshansky 🦁 — Mon, 21 Sep 2015 17:30:02 GMT

Olshansky's review of Black Mirror: Season 1

RAM is the new disk – and how to measure its performance – Part 2 – Tools

Tanel Poder Blog — Mon, 21 Sep 2015 11:20:18 GMT

[ part 1 | part 2 | part 3 ]

In the previous article I explained that the main requirement for high-speed in-memory data scanning is column-oriented storage format for in-memory data. SIMD instruction processing is just icing on the cake. Let’s dig deeper. This is a long post, you’ve been warned.

Test Environment

I will cover full test results in the next article in this series. First, let’s look into the test setup, environment and what tools I used for peeking inside CPU hardware.

I was running the tests on a relatively old machine with 2 CPU sockets, with 6-core CPUs in each socket (2s12c24t):

$ egrep "MHz|^model name" /proc/cpuinfo | sort | uniq -c
     24 cpu MHz		: 2926.171
     24 model name	: Intel(R) Xeon(R) CPU           X5670  @ 2.93GHz

The CPUs support SSE4.2 SIMD extensions (but not the newer AVX stuff):

$ grep ^flags /proc/cpuinfo | egrep "avx|sse|popcnt" | sed 's/ /\n/g' | egrep "avx|sse|popcnt" | sort | uniq
popcnt
sse
sse2
sse4_1
sse4_2
ssse3

Even though the /proc/cpuinfo above shows the CPU clock frequency as 2.93GHz, these CPUs have Intel Turboboost feature that allows some cores run at up to 3.33GHz frequency when not all cores are fully busy and the CPUs aren’t too hot.

Indeed, the turbostat command below shows that the CPU core executing my Oracle process was running at 3.19GHz frequency:

# turbostat -p sleep 1
pk cor CPU    %c0  GHz  TSC SMI    %c1    %c3    %c6 CTMP   %pc3   %pc6
             6.43 3.02 2.93   0  93.57   0.00   0.00   59   0.00   0.00
 0   0   0   4.49 3.19 2.93   0  95.51   0.00   0.00   46   0.00   0.00
 0   1   1  10.05 3.19 2.93   0  89.95   0.00   0.00   50
 0   2   2   2.48 3.19 2.93   0  97.52   0.00   0.00   45
 0   8   3   2.05 3.19 2.93   0  97.95   0.00   0.00   44
 0   9   4   0.50 3.20 2.93   0  99.50   0.00   0.00   50
 0  10   5 100.00 3.19 2.93   0   0.00   0.00   0.00   59
 1   0   6   6.25 2.23 2.93   0  93.75   0.00   0.00   44   0.00   0.00
 1   1   7   3.93 2.04 2.93   0  96.07   0.00   0.00   43
 1   2   8   0.82 2.15 2.93   0  99.18   0.00   0.00   44
 1   8   9   0.41 2.48 2.93   0  99.59   0.00   0.00   41
 1   9  10   0.99 2.35 2.93   0  99.01   0.00   0.00   43
 1  10  11   0.76 2.36 2.93   0  99.24   0.00   0.00   44

I will come back to this CPU frequency turbo-boosting later when explaining some performance metrics.

I ran the experiments in Oct/Nov 2014, so used a relatively early Oracle 12.1.0.2.1 version with a bundle patch (19189240) for in-memory stuff.

The test was deliberately very simple as I was researching raw in-memory scanning and filtering speed and was not looking into join/aggregation performance. I was running the query below with different hints and parameters to change access path options:

SELECT COUNT(cust_valid) FROM customers_nopart c WHERE cust_id > 0

So you want to work remotely...

Josh Thompson — Mon, 21 Sep 2015 03:00:00 GMT

Josh’s “rules” for getting a sweet remote job

A few weeks ago, I met a fantastic guy who is contemplating next steps for work. He is great at what he does, and is thinking about what direction to go in his life. He’s young, and thought working remotely sounded pretty cool. I agreed, and then spent the next week or two thinking about what’s required to work remotely.

Here’s the email I sent trying to explain all of this.

Hi {redacted},

Prepare yourself for a long email. I’ve had a lot of thoughts rolling around my head related to this, and you’re getting them all. I’m going to work on cleaning this up and maybe sticking it online as a blog post for later. I’ve talked with many people about working remotely, so… here are some thoughts:

Glad you got the bug for remote working!

There’s lots of good things that come with working remotely, but there are lots of real and perceived barriers.

So, let me offer some more encouragement and advice.

First, there’s the matter of framing how you go about working remotely. Obviously you want to work remotely, but put yourself in the shoes of a prospective employer. They don’t care if you work remotely or not - they want you to do great work for them and make them an effective, profitable organization.

So, if I can make up the imaginary Josh’s Rules of Getting a Remote Job, this is what they’d be:

Rule 1: Getting a remote job is exactly the same as getting a regular job, but a little harder

Companies that hire remotely tend to be younger, agile-er, and smaller. So every employee matters. Making a bad hire can be catastrophic in terms of money and opportunity cost. Your job is to make it very, very obvious that hiring YOU is a great decision. In fact, you want to be so good at what you do that passing up on you is a terrible decision.

This right here is the difference in mental approaches from an “average” job hunter and a high-performing job hunter.

The average person talks about all the reasons that they’re qualified to do the work, how quick they are to learn, how happy they are for the opportunity, etc. None of this puts a CEO/hiring manager/team lead at ease. They think:

Of course this person wants to work at my company. I have a kick-ass, quickly growing company and pay my employees well. I am looking for more than “just” a willing person.

Rule 2: A top-performing employee approaches a job opportunity the same way a consultant approaches a possible gig.

First, qualify the opportunity. Is this something that you could do well?

Next, assuming it’s a job they could do well, get into “proposal” stage.

Tell them (in as many words)

If I were hired to do this job, these are the things that I would do, these are the projects I would work on, and here’s the impact it would have on the company’s bottom line.

When you can walk someone down the path from “Hire me” to “I’ll help you make $$,$$/year more than you would otherwise”, if the the amount you’ll make them is 3x your annual salary, you’re in a pretty good place.

All of a sudden, rather than the risk being:

hire this person and it might not work out

The risk becomes becomes:

If I don’t hire this person, I might miss this opportunity to make $$$

By framing the conversation like this, you’re showing that you are aware of their needs as a business, and that you’ll hit the ground running. You don’t need your hand held, you’re able to execute ideas as appropriate.

By the way - none of this should be construed as a suggestion to “make up BS”. There are plenty of ways you can make contributions to a company and help them retain users, get new users, save money, earn more money, grow accounts, bring in new business, hire better people, keep existing employees longer, etc.

This is another topic for another time, though. Just know that you need to focus on the value you provide to a company.

Rule 3: A top-performing employee has concrete numbers and stories showing their skill.

You’ll be asked about your prior experience at your current job. Don’t just say “I did {job} at {company}”. That’s boring. That makes you a risky hire.

You’ll say something like

When I was a {position} at {company}, I was tasked with {responsibility}. Not only did I do that task well, but I took on a few side projects to increase the effectiveness of the team, and decrease the friction of some processes. For example… {story showing how you made improvements that impacted the business, with figures to back it up}.

Rule 4: Your online persona is the primary thing that will get you hired. Craft a good one.

When you see a job opportunity go by, you don’t just submit a resume and let it go. You find the person in charge of hiring, or someone on the team, or the company CEO, and you send them a short email. In that email, you make it easy for them to discover how competent and skilled you are.

That could be as simple as a link to your personal website in your signature. A personal website is extremely useful for showing a lot of things at once:

Tech savvy (it is simultaneously extremely complicated and unbelievably easy to set up a personal website. Try to get firstNameLastName.com. If not, look for firstName.com, firstNameLastName.io, firstName.io, firstNameLastName.is, etc. Look around. Find people that have strong personal brands, see what their URL is, and try to get that. (If Jekyll interests you, I wrote this guide to building a personal website in Jekyll: Build a Personal Website in Jekyll - A Detailed Guide For First-Timers)
Room to show that you can communicate effectively. Remote companies live and die on the written word. Can you communicate something interesting via text? What do you write about? It doesn’t matter, as long as you might be helping someone. Before you publish anything, answer this question: “Does what I’m about to publish have the potential to help someone?”

If the answer is yes, publish.

Showcase your non-writing skills. Do you put cool stuff on Instagram and Twitter? Well, put those on your personal website. Do you read a lot? Make book recommendations and include a reason or two for why someone should read the book. Are you athletic? There’s ways to show that too.
The skills required to work remotely tend to overlap heavily with the skills required to create a website. Software is eating the world, so get conversant in the technology.
You don’t have to go full web-developer to put a website online, just as you don’t need to be a software engineer to work at a software company. But know the tools available to you that allow you to be effective at technical tasks. The idea behind using Webflow or Squarespace to publish a website is the same as using KissMetrics or FullStory to gather customer data.

Phew. That’s it for now.

-Josh

PS All of this is general advice. You’ll need to become a specialist. That’s hard work. Holy crap, it’s hard. That is a whole other topic. But this is maybe a helpful starting point.

Want to soak in the wisdom of experts? Use https://feedly.com to subscribe to the websites of people that seem to know what they’re talking about.

Read what they think about things, and you’ll be wiser for it. :)

Links - September 21st, 2015

Avy Faingezicht — Mon, 21 Sep 2015 03:00:00 GMT

I can't recommend Gopnik's article enough. It is a really interesting read about history, faith, and human life. If you read any online article today, this should be it.

Chinatown treats review

xenodium.com @alvaro — Sun, 20 Sep 2015 03:00:00 GMT

Recommended? yep

There's a corner in Chinatown hosting some truly superb treats. If you get caught in the rush between Newport court and Newport place, you'd likely fail to notice some the awesome street food stands.

Chilly squid

I've walked past this place many times and never noticed it. They serve a handful of items, but the grilled chilly squid skewers caught my attention. They're grilled, brushed with chilly sauce and finished with sprinkled sesame and cumin seeds. Super tasty.

Pancake + Crisp + Egg + Hot chillies = Jiān Bǐng 煎餅

I first had these delicious breakfast savory pancakes at a Beijing street food stall. Never expected to randomly find Jiān Bǐng in London. It's a crepe with an additional egg spread, hoisin sauce, chilly sauce, hot chillies, topped with spring onions and coriander, all wrapping a wonderfully crispy bread cracker. And.. it's awesome.

Tai Yaki

Chinatown Bakery is hard to miss. Pedestrian traffic slows down as we all fall under the spell of the Tai Yaki machine. This wonderful assembly line produces fish-shaped sweet waffles filled with custard. They are the perfect dessert after some savory street snacks. You can get a bag of 4 for £2.

All near each other

All these delights are within a stone's throw away from each other.

Useful?

Was this post useful to you? do reply!

Better suggestion?

London is full of overhyped, gimmicky, and unnecessarily expensive restaurants. Very few deliver truly awesome food (even those expensive ones). Got suggestions? I'd love to hear from you @xenodium.

Building a MicroBus application

Daniel Little Dev — Sat, 19 Sep 2015 08:31:08 GMT

I often like to think of software architecture as the art of structuring software. It's goal being to meet requirements while at the same…

Taking up the writing habit again

Ben Overmyer's Site — Sat, 19 Sep 2015 03:00:00 GMT

A few days ago, I migrated all of my blog posts from my last primary blog on WordPress.com to this blog. In the process, I read over a lot of old posts. I noticed something interesting: back in 2007 and 2008, I was writing blog posts with fair regularity. Now, it's a rarity. The blog posts back then were fairly journal-like in character. They told a story of what I was working on, how I felt on certain subjects, and described some of my adventures in games I was playing at the time. The blog posts I wrote in the recent past trend towards the same thing when they're more frequent. When I choose to try and write something useful, like the Laravel on Docker post, it takes a lot longer, a lot more effort, and is a lot more intimidating to start on. So as I sit here sipping on my orange pekoe black tea, listening to relaxing acoustic guitar, and writing these words, I'm thinking that I should allow myself to write journal-style posts again. This, if for no other reason than to ingrain that old writing habit. It'll be easier to write the big, important stuff if the simple stuff - the writing - comes naturally, right?

2015-09-19/01

ho.dges.online — Sat, 19 Sep 2015 03:00:00 GMT

2015-09-19/02

ho.dges.online — Sat, 19 Sep 2015 03:00:00 GMT

Ed in his natural environment.

2015-09-19/03

ho.dges.online — Sat, 19 Sep 2015 03:00:00 GMT

More location and temperature stuff

Liza Shulyayeva — Fri, 18 Sep 2015 20:10:43 GMT

I realized I didn’t actually include any implementation details when writing my sleepy post about stable locations and temperature the other night. I figured I’d elaborate on that now.

React bookmarks

xenodium.com @alvaro — Fri, 18 Sep 2015 03:00:00 GMT

Links - September 18th, 2015

Avy Faingezicht — Fri, 18 Sep 2015 03:00:00 GMT

Getting to Know Fiddler: Part VIII: It isn't all rainbows and unicorns

Joshua Rogers — Thu, 17 Sep 2015 15:00:00 GMT

Roughly a year ago I posted what I expected to be the last entry in my Fiddler series. Each of the seven entries had focused on various ways that Fiddler could help improve our development life cycle. They represented the highlight reel. All isn't perfect though, and where there is a highlight reel, there tends to be outtakes as well. So, here goes: here's a few ways you can use Fiddler to slow down your development process.

Links - September 17th, 2015

Avy Faingezicht — Thu, 17 Sep 2015 03:00:00 GMT

I started a new book this week, which means my time has shifted from online consumption to staring at dead trees. More on that soon.

Walls by Urusla K Le Guin

ho.dges.online — Thu, 17 Sep 2015 03:00:00 GMT

“Like all walls, it was ambiguous, two-faced. What was inside it and was was outside it depended upon which side of it you were on.”

Urusla K Le Guin, The Dispossessed.

Bringing snail stables down to Earth

Liza Shulyayeva — Wed, 16 Sep 2015 21:28:31 GMT

SnailLife snail stables have always lived in the ether - in some virtual universe with no physical location. Since I’m trying to make SnailLife based in reality, users’ snail stables should also be based in real locations. This is why now, upon registration, the user’s physical location is used as the location of their snail stable.

Chef's Table: Season 1

🦉 olshansky 🦁 — Wed, 16 Sep 2015 16:58:44 GMT

Olshansky's review of Chef's Table: Season 1

Narcos: Season 1

🦉 olshansky 🦁 — Wed, 16 Sep 2015 16:58:38 GMT

Olshansky's review of Narcos: Season 1

Chinese rice vinegar

xenodium.com @alvaro — Wed, 16 Sep 2015 03:00:00 GMT

Note to self to buy Gold Plum Chinkiang Vinegar. Awesome with dim sum.

Links - September 16th, 2015

Avy Faingezicht — Wed, 16 Sep 2015 03:00:00 GMT

Heroku Data Links with Postgres and Redis

Brightball Articles — Wed, 16 Sep 2015 03:00:00 GMT

PostgreSQL has a great feature called Foreign Data Wrappers (FDW) that allows it to connect directly to outside systems. Although the setup can be a little complicated, once it’s available you can run queries with joins or subqueries against them, insert data, create views, etc. Heroku has dramatically simplified the process of using FDW with PostgreSQL and Redis thanks to Data Links. Let’s try it out.

PostgreSQL replication using Bucardo

Srijan Choudhary, all posts — Tue, 15 Sep 2015 21:05:00 GMT

Keeping a live replica of selected PostgreSQL tables using Bucardo

Portable Book Stand

june.kim — Tue, 15 Sep 2015 03:00:00 GMT

Links - September 15th, 2015

Avy Faingezicht — Tue, 15 Sep 2015 03:00:00 GMT

What's in a Name?

The David Brownman Blog — Tue, 15 Sep 2015 03:00:00 GMT

undefined

Read the whole thing.

Links - September 14th, 2015

Avy Faingezicht — Mon, 14 Sep 2015 03:00:00 GMT

Eigenfaces reconstructions

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Sun, 13 Sep 2015 16:53:04 GMT

While learning about “principal component analysis” for the first time, I also encountered the intriguing world of eigenfaces. Briefly put, back in the mists of time—at least in computer science terms—Matthew Turk and Alex Pentland came up with an ingenious approach towards face recognition and classification. At it’s core, the method consists of nothing but an eigenvalue decomposition. Let me briefly outline the steps:

Acquire a set of images showing faces. I am assuming that the images are grey-scale because this means that we do not have to think about different colour channels.
Treat each digital image as a vector in a high-dimensional space by “flattening” the image vector. An image of dimensions 192 ×64 pixels becomes a 12288-dimensional vector, for example.
Calculate the mean face of the data set and subtract it from all images. This corresponds to centring the distribution of faces.
Put all vectors into a matrix. The result is a matrix with as many rows as the dimensions of the faces, and as many columns as there are faces.
Calculate the eigenvalue decomposition of the sample covariance matrix. This step hides a clever trick: We do not calculate the sample covariance matrix of the original matrix, which would be a square matrix with the row dimension of the original data, for example 12288 ×12288. Instead, we calculate the transpose of this matrix, which uses the column dimension of the original data. This matrix will hence only have as many rows and columns as there are faces in the data. It can be shown that an any eigenvector of this matrix multiplied with the original matrix yields an eigenvector of the large covariance matrix.
Of course, we do not use all eigenvalues and eigenvectors but rather only as many until we have reached, say, 95% of explained variance. This removes a surprisingly large amount of data.
If we have the reduced eigensystem, we can project any new face to the (reduced) basis of eigenvectors. This allows us to embed them in what Turk and Pentland call the “face space”.
The projection permits us to decide whether something is a face at all by checking whether it is sufficiently close to the other vectors in “face space”. Furthermore, it permits face classification by assigning a known face to its nearest neighbour (or to its nearest class of faces). These operations only require basic vector multiplication operations, so they are very cheap.

Such a pretty neat algorithm deserves a neat implementation. I tried to write one myself in Python. You can find the code in the Eigenfaces repository on my GitHub profile. As always, it is licensed under an MIT license. Note that the script currently only supports the reconstruction aspect of faces. It will randomly partition input data into training and test data, perform the eigensystem calculations on the training data, and attempt the reconstruction on the test data.

I used a classical database of faces, the Yale faces data, which has been preprocessed by some nice folks at MIT. In the repository, I only included the original and cropped variants of the faces—the preprocessed data set contains much more.

The results are interesting and highlight some of problems with the approach. Since the eigendecomposition is completely agnostic concerning any sorts of facial features, it may “learn” the wrong thing. Input data needs to be chosen carefully and should ideally be already cropped. Otherwise, the reconstruction quality is not very great. For example, this happens when I train on 100 out of 165 images with 95% explained variance:

We can see that our reconstruction (left) captures mostly the uninteresting stuff, namely the position of the head, as well as some lighting details, in comparison to the original (right). Here”s a similar example with the cropped faces:

This version is much better. Of course, it also contains the ghostly remains of the glasses, but all in all, the reconstruction is more true to the original.

For such a conceptually simple algorithm, these are rather nice results. It is very intriguing to see how well even simple classifiers can perform. If you want to play with the algorithm yourself, please grab a copy of the repository.

A walk from the subway station — inspired by The Hunger Games

🦉 olshansky 🦁 — Sun, 13 Sep 2015 09:12:08 GMT

Prior to my senior year in College, I made my way through the Hunger Games Trilogy. I read all three books back-to-back in a relatively…

Use ImageMagick to batch-resize images

xenodium.com @alvaro — Sun, 13 Sep 2015 03:00:00 GMT

Using percentage:

$ mogrify -resize 10% *.png

Using dimensions:

$ mogrify -resize 120x120 *.png

Lots of other alternatives from ImageMagick's documentation:

-resize scale% -resize scale-x%xscale-y% -resize width -resize xheight -resize widthxheight -resize widthxheight^ -resize widthxheight! -resize widthxheight> -resize widthxheight< -resize area@

Fix image aspect ratios for Instagram:

$ mogrify -resize 1080x1350 -gravity center -extent 1080 *.jpg

Trip Report: New River Gorge

Josh Thompson — Sun, 13 Sep 2015 03:00:00 GMT

Kristi and I are spending a few weeks in Fayetteville, WV, home of the New River Gorge. There’s fantastic climbing here. I climbed with good friends, and was absolutely humbled by how strong they all are. (My defense, at least for the next few weeks, is that I’ve not climbed consistently for 4.5 months)

I’ve been climbing regularly this last week, trying to recondition my skin and muscles. I’ve got a long list of projects to work on (thanks Brian and Christian) so the next few weeks should be productive.

I got on a lot of hard climbs, and made it to the actual tops of few of them. Sent nothing. Lots of room for improvement.

Walking through the rainy woods to climb.

Anvil: move fast and break things

zserge's blog — Sun, 13 Sep 2015 03:00:00 GMT

Dear users, those who were early followers of Anvil. I’m now heavily working on the next version of Anvil, but it’s likely to make you rewrite your code, and I’m deeply sorry about that. Below I will describe why the changes are necessary and I hope you will find that your code would not be modified a lot and embrace the benefist it brings. how anvil works now Currently we have Renderables which have a view() method returning a virtual layout.

SSD Storage - Two Months In Production

smcleod.net — Sun, 13 Sep 2015 03:00:00 GMT

Over the last two months I’ve been running selected IO intensive servers off the the SSD storage cluster, these hosts include (among others) our:

Primary Puppetmaster
Gitlab server
Redmine app and database servers
Nagios servers
Several Docker database host servers

Reliability

We haven’t had any software or hardware failures since commissioning the storage units.

During this time we have had 3 disk failures on our HP StoreVirtual SANs that have required us to call the supporting vendor and replace failed disks.

Aloha

🦉 olshansky 🦁 — Sat, 12 Sep 2015 10:46:48 GMT

Olshansky's review of Aloha

Citizenfour

🦉 olshansky 🦁 — Sat, 12 Sep 2015 10:39:06 GMT

Olshansky's review of Citizenfour

Orange Is the New Black: Season 3

🦉 olshansky 🦁 — Sat, 12 Sep 2015 10:35:11 GMT

Olshansky's review of Orange Is the New Black: Season 3

Nintendo PowerFest (1994) Reunion Tournament

KernelCurry — Sat, 12 Sep 2015 03:00:00 GMT

A few weekends ago, I flew into Arizona to attend the Game On Retro Expo with some old friends. Among this group was Dorion Whitlock (1994 Nintendo PowerFest finalist and old roommate) and Mike Iarossi (1994 Nintendo PowerFest first place winner). To say the least, I forgot how much fun it is playing video games with your friends.

The main draw for the Game On Retro Expo was the 1994 PowerFest Reunion Tournament. Everyone at the expo was able to play a reproduction of the 1994 champion cartridge in hopes of qualifying for the Reunion tournament on Sunday. This cartridge was time capped at about six minutes and consisted of the lost level in Mario, five laps in Mario Kart, and Ken Griffey Jr. Home Run Derby. The top ten scoring players of the weekend qualified and the winner of the tournament played Mike Iarossi, the 1994 champion, on the real 1994 cartridge.

The Lanyard By Billy Collins

ho.dges.online — Sat, 12 Sep 2015 03:00:00 GMT

The other day I was ricocheting slowly
off the blue walls of this room,
moving as if underwater from typewriter to piano,
from bookshelf to an envelope lying on the floor,
when I found myself in the L section of the dictionary
where my eyes fell upon the word lanyard.

No cookie nibbled by a French novelist
could send one into the past more suddenly -
a past where I sat at a workbench at a camp
by a deep Adirondack lake
learning how to braid long thin plastic strips
into a lanyard, a gift for my mother.

2015-09-12

ho.dges.online — Sat, 12 Sep 2015 03:00:00 GMT

Moody me.

Links - September 11th, 2015

Avy Faingezicht — Fri, 11 Sep 2015 03:00:00 GMT

Get Going with Laravel on Docker

Ben Overmyer's Site — Fri, 11 Sep 2015 03:00:00 GMT

This post was updated May 14th, 2016 to significantly rework the tutorial. Docker is slowly taking over the world of web infrastructure. It makes working with multiple different services easy, and the problem of “works in dev, not in prod” goes away, since you have the same environment on your local machine as you do in the production infra. It also makes things like trying out web apps without deploying them to servers really easy. Ever wanted to just check out a personal demo of, say, WordPress or Ghost? Docker makes that simple. Docker can be intimidating to start out with. It's a complex beast, but once you've gotten it set up a couple times, it'll become second nature. This article will walk through the entire process of having a completely new and fresh Mac OS X environment all the way to running Docker container with your own Laravel application. It's a long tutorial, so grab a big mug of coffee and put on your favorite track and let's get to work. Note: This tutorial is for OS X. A lot of the pre-req stuff is different for Windows, so if you're running that OS, you might be better off using Docker Toolbox. Until the new Docker App comes out, anyway.

Prerequisites

The first step is getting some basic tools installed. OS X comes with git and Ruby installed, but we need to make a slight adjustment to our Ruby environment, and we'll need Homebrew to install some other things.

Homebrew

Let's start with Homebrew. Paste this into a terminal prompt:

ruby -e "$(curl -fsSL https://raw.githubusercontent.com/Homebrew/install/master/install)"

Running that will install Homebrew. Next, we'll need Homebrew Cask:

brew install caskroom/cask/brew-cask

This is used to install binary applications that otherwise you'd use a GUI for.

Virtualbox

Let's install this via Homebrew Cask to make things easy.

brew cask install virtualbox

Enter Docker

Now we get to the meat of it - installing Docker and the other tools necessary to work with containers. We're going to be using a wrapper called dinghy for this. You could use the official Docker Toolbox, but the default VM that it provides, boot2docker, relies on vboxfs, which makes real-time updates (like gulp watch) really slow.

Install Docker Toolbox

When I originally wrote this post, Docker Toolbox made some questionable assumptions about how people were using Docker for development. Those have now been corrected, and it's safe to go ahead and install Docker Toolbox. You can get that here: https://www.docker.com/products/docker-toolbox

Docker is installed! Now what?

Now everything is all set up and ready to get container-ing. In the next section, we'll create a basic Laravel app to get a proper Docker workflow going.

Getting set up for Laravel

For this tutorial, we're going to use the Laravel framework. This will demonstrate how to use multiple containers while not requiring us to write a ton of code to get started.

Install Composer

PHP is installed by default with OS X, but you're going to need Composer also. Thankfully, it's a quick install:

curl -sS https://getcomposer.org/installer | sudo php -- --install-dir=/usr/local/bin --filename=composer

Make sure that composer is in your PATH in your bash/zsh profile. Look for this line: ~/.composer/vendor/bin If it's nowhere in the file, you'll need to add something like this near the bottom of the file:

export PATH="$PATH:~/.composer/vendor/bin"

Install the Laravel CLI tool

This tool makes starting new Laravel projects much easier and faster:

composer global require laravel/installer

Start a new Laravel project

In a directory that makes sense for you (I use ~/Source/ for my code projects), run this shell command:

laravel new docker-app

This will create a new Laravel project called “docker-app” in a new docker-app directory. You're now ready to Docker-ize this app!

Docker-ize!

The next step is to create four files that we'll use to describe our container environment. These files are nginx-vhost.conf, supervisord.conf, Dockerfile, and docker-compose.yml. Each of these is at the root of your project. The contents of each file is as follows:

nginx-vhost.conf

We're using nginx as the web server for this app. We could use Apache, but for this exercise, we'll stick with nginx. The below file configures the virtualhost that will expose our Laravel app to the container.

server {
 listen 80;
 listen [::]:80 default ipv6only=on;
 root /srv/www/public;
 index index.php;

 # Disable sendfile as per https://docs.vagrantup.com/v2/synced-folders/virtualbox.html
 sendfile off;

 # Add stdout logging
 error_log /dev/stdout info;
 access_log /dev/stdout;

 location / {
 try_files $uri $uri/ /index.php$is_args$args;
 }

 location ~ .php$ {
 try_files $uri =404;
 fastcgi_split_path_info ^(.+.php)(/.+)$;
 fastcgi_pass unix:/var/run/php5-fpm.sock;
 fastcgi_param SCRIPT_FILENAME $document_root$fastcgi_script_name;
 fastcgi_param SCRIPT_NAME $fastcgi_script_name;
 fastcgi_index index.php;
 include fastcgi_params;
 }

 location ~* .(jpg|jpeg|gif|png|css|js|ico|xml)$ {
 expires 5d;
 }
 # deny access to . files, for security
 location ~ /. {
 log_not_found off;
 deny all;
 }
}

supervisord.conf

We're using Supervisor to run nginx and PHP-FPM. It makes things a little cleaner and easier. Below is the config file for Supervisor.

[supervisord]
nodaemon=true

[program:php5-fpm]
command=/usr/sbin/php5-fpm
autostart=true
autorestart=true
priority=5

[program:nginx]
command=/usr/sbin/nginx
autostart=true
autorestart=true
priority=10
stdout_events_enabled=true
stderr_events_enabled=true

Dockerfile

For a guide to the syntax you see below, check out the official Docker documentation for Dockerfiles.

FROM ubuntu:14.04
ENV DEBIAN_FRONTEND noninteractive
RUN apt-get update -y
RUN apt-get install -y software-properties-common
RUN add-apt-repository ppa:nginx/development
RUN apt-get update -y
RUN apt-get upgrade -y
RUN apt-get install -y supervisor nginx php5-fpm php5-cli php5-curl php5-gd php5-mysql php5-memcached php5-mcrypt

# Clean up to reduce container size
RUN apt-get remove --purge -y software-properties-common
RUN apt-get autoremove -y
RUN apt-get clean
RUN apt-get autoclean
RUN echo -n > /var/lib/apt/extended_states
RUN rm -rf /var/lib/apt/lists/*
RUN rm -rf /usr/share/man/??
RUN rm -rf /usr/share/man/??_*

# Configure php-fpm to not run as a daemon
RUN sed -e 's/;daemonize = yes/daemonize = no/' -i /etc/php5/fpm/php-fpm.conf
RUN sed -e 's/;listen.owner/listen.owner/' -i /etc/php5/fpm/pool.d/www.conf
RUN sed -e 's/;listen.group/listen.group/' -i /etc/php5/fpm/pool.d/www.conf

# Configure nginx to not run as a daemon
RUN echo "daemon off;" >> /etc/nginx/nginx.conf

# This next line lets nginx write to your working directory
RUN usermod -u 1000 www-data

# Configure nginx virtualhost
RUN rm -Rf /etc/nginx/conf.d/*
RUN rm -Rf /etc/nginx/sites-available/default
ADD ./nginx-vhost.conf /etc/nginx/sites-available/default.conf
RUN ln -s /etc/nginx/sites-available/default.conf /etc/nginx/sites-enabled/default.conf

# Add the application code into the container
ADD . /srv/www

# Configure Supervisor
ADD ./supervisord.conf /etc/supervisor/conf.d/supervisor.conf

# Fix permissions
RUN chown -Rf www-data:www-data /srv/www/

# Set our working directory
WORKDIR /srv/www

# Expose Ports
EXPOSE 80

# And finally, run the command to kickstart everything
CMD ["/usr/bin/supervisord"]

The above file is telling Docker how to build the container. It includes nginx and PHP-FPM, but it leaves database and cache store to other containers. The container's main job is to serve the Laravel application at /srv/www in the container on port 80 using nginx and PHP-FPM.

docker-compose.yml

This file is what Docker Compose will use to build our stack and link up the containers the app needs to function. It defines three containers - web, db, and cache. The “volumes” part of the web container's definition here will map your source code directory on your host to the container's app directory, which will let you update your container by editing files on your host without having to rebuild the container. Under the db container, you'll see the “environment” section. This is a list of environment variables that will be set in the MySQL container. The two blank ones will use the environment variable of the same name on your host to set them inside the container, which is far more secure than putting these values into version control! Just make sure that you set them! That'll happen in your bash/zsh profile, with lines like this:

export MYSQL_ROOT_PASSWORD=supersecurepassword1
export MYSQL_PASSWORD=anothersecurepassword2

Alright. Without further ado, here's the docker-compose.yml file:

web:
 build: .
 ports:
 - "8080:80"
 volumes:
 - .:/srv/www
 links:
 - db
 - cache
db:
 image: mysql:5.6
 environment:
  MYSQL_ROOT_PASSWORD:
  MYSQL_DATABASE: dockerapp
  MYSQL_USER: dockerapp
  MYSQL_PASSWORD:
cache:
 image: memcached:latest

Configure your Laravel app

Laravel 5.0 and later uses a .env file to configure the app. This is where you'll give your app the settings it needs to talk to the other services (in this case, memcached and MySQL).

.env

APP_ENV=local
APP_DEBUG=true
APP_KEY=dhBKg8eSVZ3tH5yuz6z40j13X13YWTJf

DB_HOST=db
DB_DATABASE=dockerapp
DB_USERNAME=dockerapp
DB_PASSWORD=anothersecurepassword2

CACHE_DRIVER=memcached
SESSION_DRIVER=memcached
QUEUE_DRIVER=sync

MEMCACHED_HOST=cache

MAIL_DRIVER=smtp
MAIL_HOST=mailtrap.io
MAIL_PORT=2525
MAIL_USERNAME=null
MAIL_PASSWORD=null
MAIL_ENCRYPTION=null

Run your new application container

Alright, we're just about ready for the finale. First we need to run Docker Compose against our app to get it running.

docker-compose build

This will build your container and fetch the other two from Docker Hub. It shouldn't take too long - the longest part will be downloading mysql and memcached containers. Then, put your containers online:

docker-compose up

Now try visiting http://192.168.99.100:8080 in your browser of choice. You might have to change the IP address to match that of your Docker machine (docker-machine ip default will give you that). You should see the Laravel welcome screen, and your work here is complete. Note: don't worry that your command prompt didn't come back. This runs in the foreground, so you'll need to use a new tab in your terminal app for other things. Or, you can use the daemon mode instead:

docker-compose up -d

Congratulations, you're now working with Docker!

V3 Technology Awards: Vote for Canonical!

kyrofa's blog — Fri, 11 Sep 2015 03:00:00 GMT

I’m pleased to announce that Canonical has made the shortlist in several categories for the V3 Technology Awards! Now it’s all in your hands: Go and Vote for Canonical! Note that voting closes at 5pm on Friday, October 9th. The categories in which we made the shortlist are: Best cloud computing provider We made this as an overall company: Canonical Best business software provider We also made this as an overall company: Canonical Best Internet of Things product Made this as Canonical Snappy Ubuntu Core Thank you for your support!

Pesaro, Microsoft and OpenOffice: the consequences

Welcome to Marco Fioretti's website! on Stop at Zona-M — Thu, 10 Sep 2015 09:03:11 GMT

A few days ago I summarized the most questionable or uncertain points of the software odissey of the City of Pesaro, saying that I’d also post questions and consequences, both for the City and Open Source advocates, not mentioned yet in this story. For Pesaro, the road forward has little or nothing to do with the initial topic, that is Open Source Software in Public Administration. The advocates, instead, should rethink some of their strategies. Let’s start from Pesaro, but what follows applies to practically every city.

Moonwalking With Einstein

june.kim — Thu, 10 Sep 2015 03:00:00 GMT

Thinking, Fast and Slow

june.kim — Thu, 10 Sep 2015 03:00:00 GMT

Links - September 10th, 2015

Avy Faingezicht — Thu, 10 Sep 2015 03:00:00 GMT

The bias of list creation in the form of class methods

nutcroft — Thu, 10 Sep 2015 03:00:00 GMT

It's interesting how human biases carry on to writing code and how they evolve in this transition.

Recently, at my job, we wanted to create a metrics endpoint. In order to do this task, we copy-pasted the controller implementation from the users-list endpoint to the metrics one.

Of course, after using such a technique we had to comb through the users-list code and check for possible edits or removal of parts of code that are not useful for the metrics functionality.

However, there is a psychological phenomenon that caused us hours tracking a bug! In this phenomenon, given a list and being asked to divide, depending on your way of operation, different sets of lists will be created, even with the same comparing function.

To illustrate this, let us assume you have a list and someone asks you to create a new list with the most important items.

You have two ways of operating:

Create an empty new list and check out every item of the original list. If it is important, copy it into the new list.
Duplicate the original list. Check out every item of the new list, and if it not important remove it.

If you pick the first way, you will have a shorter important list than the second way. The reason is that on those items that you are torn whether they are important, you will act with the default action. In the first case, the default is not copying them to the new list. In the second case, the default action is not removing them from the new list.

A way which wouldn't cause a bias is one with no default action. In this case, it would be to create two new lists. One is the unimportant items and the other the important ones. Subsequently, there is no default aka easier act; either you write the item in the first list or in the second.

This phenomenon applies to the way we wrote the new endpoint controller. Essentially, the list was the list of methods in the controllers and the way we operated was the second one, using duplication.

As a result, the new list of methods was larger. The default action of not removing code/methods that might be useful later was followed. One of them was to keep the pagination functionality enabled. However, the dummy data set in our local installation was smaller than the production one. As a result, it wasn't possible to get new metrics summaries in the production after a few, because the request was actually only for page 1. We had to dive through the stack, front, back layers and check database durability (sorry Mongo) in order to locate this extra method at the end.

Links - September 9th, 2015

Avy Faingezicht — Wed, 09 Sep 2015 03:00:00 GMT

OpenConfig Interfaces - Some Examples

rob.sh — Wed, 09 Sep 2015 02:23:00 GMT

I’ve talked a little on this site before about what we’re trying to achieve with OpenConfig. However, one of the observations that it’s easy to make is that YANG models alone don’t really achieve anything in terms of making the network more programmable. To make the network more programmable, we need to have tooling that helps us create instances of those modules, manipulate them, and then serialise the into a format that can be used to transmit data that conforms to the model to a device.

Lucky 7 review

xenodium.com @alvaro — Tue, 08 Sep 2015 03:00:00 GMT

Recommended? yep

Lucky 7 is a small nostalgic American diner on Westbourne Park road. I like the vibe, the space, and the unpretentious waiting staff. I go to Lucky 7 often enough, originally for the buttermilk pancakes, but the list of favorites on the menu keeps growing.

Smileys by w.dyer.

Buttermilk Banana pancakes

These are my favorite pancakes in London by far. Banana buttermilk pancakes and a few free coffee refills usually sort me out until dinner time. Add a side of bacon if extra hungry. You probably don't need it though.

Reuben sandwich

The reuben has been on Lucky 7's specials menu for months now. Not had many of these in London, but compared to The Brass Rail's, this reuben was a clear winner. The sandwich is huge and comes with fries. My girlfriend and I struggled to finish one between the two of us.

Vanilla milkshake (add malt!)

This milkshake hits the spot every time, but it's filling. You almost have to decide between the shake and an actual meal. If you must have it, add malt. Sorry, no picture.

Huevos Rancheros

This is a breakfast dish I can equally make (better?) at home, but Lucky 7 wins hands down every time I'm feeling particularly lazy. Sorry, no picture.

Useful?

Was this post useful to you? do reply!

Better suggestion?

London is full of overhyped, gimmicky, and unnecessarily expensive restaurants. Very few deliver truly awesome food (even those expensive ones). Got suggestions? I'd love to hear from you @xenodium.

Links - September 8th, 2015

Avy Faingezicht — Tue, 08 Sep 2015 03:00:00 GMT

Managing bash aliases

Thought Eddies — Tue, 08 Sep 2015 00:05:00 GMT

Managing bash aliases

Microsoft vs OpenOffice in Pesaro: first, let's recap

Welcome to Marco Fioretti's website! on Stop at Zona-M — Mon, 07 Sep 2015 06:38:04 GMT

Pesaro is a town of about 100 thousands people on the northern adriatic coast of Italy. Its Public Administration has been facing lots of critics from Free/Open Source software supporters because, in the last five years, it changed twice the same, important part of its ICT infrastructure. Both those changes bring consequences and open issues, both for the critics and for Pesaro, that have had little or no coverage at all so far, especially outside Italy (1). Before talking about them, however, it is necessary to summarize what happened.

A Pattern Language

june.kim — Mon, 07 Sep 2015 03:00:00 GMT

Links - September 7th, 2015

Avy Faingezicht — Mon, 07 Sep 2015 03:00:00 GMT

I spent most of last week moving to my new apartment in San Francisco, which meant I had no time to post links. Here's a big list to make up for a lost week:

60 hours in Edinburgh (festival time)

Jonathan Jones homepage: planetjones.net — Mon, 07 Sep 2015 03:00:00 GMT

A 60 hour visit to Edinburgh, our old hunting ground during the August festival season.

Elixir binary search

Thought Eddies — Sun, 06 Sep 2015 22:08:00 GMT

Elixir binary search

What do USA special forces and Mexican drug lords have in common?

Welcome to Marco Fioretti's website! on Stop at Zona-M — Sun, 06 Sep 2015 08:24:27 GMT

Apparently, they both don’t know enough of how the Internet and digital (meta) data work.

Why Automate Code Reviews?

Dan Stroot — Sun, 06 Sep 2015 03:00:00 GMT

Ask a programmer to review 10 lines of code, they'll make 10 suggestions. Ask them to review 1,000 lines of code and they'll say "it looks good!"

This post Why Automate Code Reviews? first appeared on Dan Stroot's Blog

The Free Software icon that we need the most. Fifteen years ago

Welcome to Marco Fioretti's website! on Stop at Zona-M — Sat, 05 Sep 2015 19:51:00 GMT

Today, while cleaning up old backups, I found a text file named as this post, which I saved on November 17th, 2000, but never used. Cannot remember what I was planning to do with it, but here it goes. A bit naive, surely dated (just look at which Free Software companies I was suggesting to go for help…) but still interesting, considering how things stand today. Here it goes, unchanged:

Snail stable breeding brands

Liza Shulyayeva — Sat, 05 Sep 2015 11:18:05 GMT

I haven’t made a SnailLife update in a while, but I had a good reason! For about half a month I’ve been working on an entry into the js13kGames challenge. I’ve entered the challenge annually since it started and this year was meant to be no exception. Unfortunately…I wasn’t really into it this year. It felt like work, and with things at actual work being so busy I found myself not having the time or motivation to sit down and work on it after I get home.

Surely You're Joking, Mr. Feynman

june.kim — Fri, 04 Sep 2015 03:00:00 GMT

Benchmarking Go in the cloud

zserge's blog — Fri, 04 Sep 2015 03:00:00 GMT

Benchmarking Go in the cloud Go now seems to be a natural choice for writing your next microservice or web app. It’s mature and fast. But where would you host your app? In this post I will compare some popular PaaS (platform-as-a-service) and how the same primitive Go web app performs on each of them. Go I’m using the latest Go compiler and the latest version of Gin framework with zero-allocation router.

Missionary Service

Posts on Sean Lane — Fri, 04 Sep 2015 03:00:00 GMT

I spent two years in South America where I served as a Missionary for The Church of Jesus Christ of Latter-Day Saints in Asunción, Paraguay

My missionary service in Paraguay has been one of the defining experiences in my life. I’m very grateful for the opportunity to have met some amazing people and share a message that means the world to me. Below are some albums from different areas where I resided over the course of my two year mission.

Mission Area	Duration
Puente Kyjha, Canindeyu	March - June 2010
Aratiri, Capiatá, Central	June - December 2010
Mariano Roque Alonso, Central	December 2010 - January 2011
Campo Grande, Asunción, Distrito Capital	January - July 2011
Horqueta, Concepción	July - November 2011
Tayuasape, San Lorenzo, Central	November 2011 - February 2012

[Computing] On Nexuses

devever.net/~hl — Fri, 04 Sep 2015 03:00:00 GMT

Nexuses are an underrecognised utility in computing, despite the fact that they are commonly used and absolutely essential to usable computer systems. A nexus, broadly, is some sort of infrastructural system which is consumed by many different applications written by many different people. Many of these might be called “platforms”, but there are plenty of nexuses which would not be called platforms.

ZLog asynchronous I/O support

Noah Watkins — Fri, 04 Sep 2015 03:00:00 GMT

Bringing zlog into the future with a new asynchronous interface.

Building a Redis clone in Haskell

Honza Pokorný — Thu, 03 Sep 2015 18:00:00 GMT

In this post, we will attempt to make a simplified clone of Redis in Haskell. Here is a set of requirements that we will aim to fullfill:

get and set operations
Multi-threaded
Atomic
Redis compatible (implement the Redis protocol)

We should be able to use the redis-cli tool to connect to our server and issue commands to it.

We are going to omit many features that Redis has. For example, there will be no disk persistence. We will accomplish this with about 100 lines of Haskell.

Getting started: stack

We are going to use stack to build our project. Stack is a new build tool for Haskell projects. We can also use it to create all the necessary files that make up a Haskell project. You can find the installation instructions here.

Let’s create our project. We will call our server Redish.

$ stack new Redish simple

This will create a directory Redish/ with a few files in it.

Redish/
    LICENSE
    README.md
    Redish.cabal
    Setup.hs
    src/
        Main.hs
    stack.yaml

You can use stack to build this project and run it:

$ stack build
Redish-0.1.0.0: configure
Configuring Redish-0.1.0.0...
Redish-0.1.0.0: build
Preprocessing executable 'Redish' for Redish-0.1.0.0...
[1 of 1] Compiling Main             ( src/Main.hs, .stack-work/dist/x86_64-osx/Cabal-1.22.4.0/build/Redish/Redish-tmp/Main.o )
Linking .stack-work/dist/x86_64-osx/Cabal-1.22.4.0/build/Redish/Redish ...
Reidhs-0.1.0.0: install
Installing executable(s) in
/Users/<user>/<dirs>/Redish/.stack-work/install/x86_64-osx/lts-3.2/7.10.2/bin
$ stack exec Redish
hello world

Types

Let’s start by defining our types. Redish is an in-memory database so we will need a representation of our database. For a simple key-value store, all that we need is a simple map. Let’s create a few aliases.

type Value = ByteString
type Key   = ByteString
type DB    = Map Key Value

Next, we will need to represent the commands that our server knows how to handle. The command data structure can be a get, a set or unknown.

data Command = Get Key
             | Set Key Value
             | Unknown
             deriving (Eq, Show)

Software transactional memory

Now that we have our types in places, we need to write a few functions to operate on them. We need a way to insert data and to query our database.

First things first though. Since by default everything in Haskell is immutable, how can we change the value of our in-memory database? We can’t simply overwrite the old value with the new one. The compiler won’t let us. The answer is software transactional memory, or STM for short.

STM allows us to atomically change a value in our program. The atomic part is important. Many parts of the code can update this value and we have no way of knowing when and how often they might do so. STM allows us to perform atomic updates. This way any updates to our database will be run sequence even when coming from different threads. The only cost is that we have to perform any updates within the context of IO.

Our DB type will be become TVar DB. The TVar type represents the mutable reference. Next, let’s create the initial value in the main function:

main :: IO ()
main = do
    database <- atomically $ newTVar $ fromList [("__version__", "0.1.0")]

This will create a Map with a key of __version__ which has the value 0.1.0. Then, it wraps that Map in a TVar and atomically assigns it to the database variable. Each time we want to write or read the database value, we have to use IO. Let’s create a helper for atomically reading this value:

atomRead :: TVar a -> IO a
atomRead = atomically . readTVar

And let’s make a function to update a value in the database. This takes a function that does the updating and runs it through the STM machinery.

updateValue :: (DB -> DB) -> TVar DB -> IO ()
updateValue fn x = atomically $ modifyTVar x fn

Reply parsing

Next, let’s talk about the Redis protocol. It’s a simple TCP scheme that looks like this:

*2\r\n$3\r\nget\r\n$4\r\nname

It’s a bunch of keywords and arguments separated by newlines. If we clean it up and break each thing to its own line, we get:

*2
$3
get
$4
name

Let’s look at each line. *2 says to expect a command that has two arguments. $3 says that the first argument is three characters long. get is the three-character argument from above. $4 is the length of the second argument, and name is the value of the second argument. If you’re in the REPL provided by redis-cli, and you type get name, Redis will translate those two words into the above representation. A set command would look like this:

*3
$3
set
$4
name
$5
honza

This is what will be sent when you run set name honza.

This multi-argument message is called multibulk in the Redis documentation. There are two other data types that Redis uses that will interest us: the OK and the error.

When Redis needs to tell you that it accepted request and everything went smoothly, it simply responds with +OK. When Redis needs to indicate an error, it replies with -ERR something went wrong (where “something went wrong” is the message).

This format is very simple and actually very effective. When we listen on a socket for incoming messages, we have a look at the very first character. + tells us that it’s OK, - signals and error, and * tells us to keep reading for commands. We incrementally read from the socket only as much data as the protocol tells us.

In this section, we will write a parser for multibulk messages. We will use the amazing attoparsec library for this.

The following code is heavily influenced by the Hedis library. Credit goes to Falko Peters. Thanks!

A multibulk message is called a reply in Redis lingo. Let’s make a type for it.

data Reply = Bulk (Maybe ByteString)
           | MultiBulk (Maybe [Reply])
           deriving (Eq, Show)

A Bulk reply is a simple string like get or name above. MultiBulk is the whole message. Let’s also write a function that attempts to convert a Reply to a Command.

parseReply :: Reply -> Maybe Command
parseReply (MultiBulk (Just xs)) =
  case xs of
    [Bulk (Just "get"), Bulk (Just a)]                -> Just $ Get a
    [Bulk (Just "set"), Bulk (Just a), Bulk (Just b)] -> Just $ Set a b
    _                                                 -> Just Unknown
parseReply _ = Nothing

Next, let’s use attoparsec to write a parser for the Reply data type.

replyParser :: Parser Reply
replyParser = choice [bulk, multiBulk]

Our replyParser will try to match either a bulk or a multiBulk. Let’s implement those:

bulk :: Parser Reply
bulk = Bulk <$> do
    len <- char '$' *> signed decimal <* endOfLine
    if len < 0
        then return Nothing
        else Just <$> take len <* endOfLine

multiBulk :: Parser Reply
multiBulk = MultiBulk <$> do
    len <- char '*' *> signed decimal <* endOfLine
    if len < 0
        then return Nothing
        else Just <$> count len replyParser

First, the parsers look at the first character to see what kind of message it is. If it starts with a $, it’s a bulk. If it starts with a *, it’s multibulk. Then, it reads as many characters from the input as the length indicator said. In the case of multibulk, it recurses because it can contain bulk messages.

You can now run:

> parse replyParser "*2\r\n$3\r\nget\r\n$4\r\nname\r\n"
> (MultiBulk (Just [(Bulk (Just "get")), (Bulk (Just "name"))]))

Networking

At this point, we have our data structures ready and we know how to parse incoming requests into them. Now we need to work on the networking part. Let’s teach our program how to listen on a socket and parse incoming text into something useful.

Let’s change our main function to this:

main :: IO ()
main = withSocketsDo $ do
    database <- atomically $ newTVar $ fromList [("__version__", version)]
    sock <- listenOn $ PortNumber 7777
    putStrLn "Listening on localhost 7777"
    sockHandler sock database

This is pretty straight-forward. Ask for a socket and then listen on it. When something happens on the socket, run the function socketHandler. Let’s implement that next:

sockHandler :: Socket -> TVar DB -> IO ()
sockHandler sock db = do
    (handle, _, _) <- accept sock
    hSetBuffering handle NoBuffering
    hSetBinaryMode handle True
    _ <- forkIO $ commandProcessor handle db
    sockHandler sock db

Given a socket and a reference to a mutable database, we can get a handle and start processing requests. For each new connection, run forkIO which will do all this work of parsing and responding on a new lightweight thread. At the end, we simply recurse to accept new work. The commandProcessor function does the heavy lifting here, so let’s write that next.

commandProcessor :: Handle -> TVar DB -> IO ()
commandProcessor handle db = do
    reply <- hGetReplies handle replyParser
    let command = parseReply reply
    runCommand handle command db
    commandProcessor handle db

This function runs the replyParser we wrote earlier. It uses a very clever function called hGetReplies which we will look at in a minute. It will read as much data as necessary from the handle to get an instance of Reply. We then convert that reply to a command and run it.

hGetReplies :: Handle -> Parser a -> IO a
hGetReplies h parser = go S.empty
  where
    go rest = do
        parseResult <- parseWith readMore parser rest
        case parseResult of
            Fail _ _ s   -> error s
            Partial{}    -> error "error: partial"
            Done _ r     -> return r

    readMore = S.hGetSome h (4*1024)

The parseWith function does partial matching. When it can’t parse anything, it will use the readMore function to get more data and try again. The readMore function uses the handle to read more data. Once the parser can match something, we are done.

Running commands

Once we have a command, we can run it!

runCommand :: Handle -> Maybe Command -> TVar DB -> IO ()
runCommand handle (Just (Get key)) db = do
    m <- atomRead db
    let value = getValue m key
    S.hPutStr handle $ S.concat ["$", valLength value, crlf, value, crlf]
        where
            valLength :: Value -> ByteString
            valLength = pack . show . S.length
runCommand handle (Just (Set key value)) db = do
    updateValue (insert key value) db
    S.hPutStr handle ok
runCommand handle (Just Unknown) _ =
    S.hPutStr handle $ S.concat ["-ERR ", "unknown command", crlf]
runCommand _ Nothing _ = return ()

When the command is a get, we read the DB atom. Then we construct a bulk reply and write it to the handle. The bulk reply is in the same format as our messages above: $5\r\nhonza\r\n. The getValue function is a lookup function that returns “null” if a value can’t be found.

getValue :: DB -> Key -> Value
getValue db k = findWithDefault "null" k db

When the command is a set, we use our updateValue function from above and write the ok to the handle. The ok variable is just +OK\r\n.

When the command is unknown, we write an error to the handle.

Compiling and running

You can now build your program with

$ stack build

And run it with

$ stack exec Redish
Listening on localhost 7777

To test it out, you can connect to it with the redis-cli tool:

$ redis-cli -p 7777
127.0.0.1:7777> set name honza
OK
127.0.0.1:7777> get name
"honza"

You can test the performance with something silly, like:

$ time redis-cli -r 10000 get name

Conclusion

You can see the finished product on GitHub. Feedback is welcome, so are questions.

Eliminate Useless Callback Wrappers in JavaScript

Pete's Points — Thu, 03 Sep 2015 07:21:37 GMT

Just a quick tip from some code I encountered today. Instead of this:

function outerFunc (callback) {
  doSomething(42, function (err) {
    callback(err)
  })
}

Eliminate the useless callback wrapper function:

function outerFunc (callback) {
  doSomething(42, callback)
}

It's more concise and more efficient and entirely equivalent.

Can we really trust the official Docker images?

Erethon's Corner — Thu, 03 Sep 2015 03:53:22 GMT

Yeap, this is another rant on the security practices of Docker users. Like we didn't have enough of those already.

lua-sh: calling shell commands as functions

zserge's blog — Thu, 03 Sep 2015 03:00:00 GMT

Lua is one of my favourite languages. It’s tiny, it’s fast, it has simple grammar and is very easy to learn. I also often write shell scripts - from simple one-liners, to bigger ones containing business logic and binding together smaller app components. In fact, this blog is powered by a few shell scripts to generate list of posts, rss xml etc. I had an experience in the past when Bash script became hard to maintain.

Eagle Scout

Posts on Sean Lane — Thu, 03 Sep 2015 03:00:00 GMT

After 7 years in scouting, I achieved the rank of Eagle Scout. My project consisted of removing an existing flagpole that was too small and installing a custom made one that was twice as large. The flagpole sits adjacent to a memorial to fallen aerial firefighters outside of the Porterville Air Attack Base in Porterville, California.

An album of the project and a small article from the local newspaper about the project may be viewed at the following links:

Go Forth and Search

Ben E. C. Boyter — Thu, 03 Sep 2015 01:25:18 GMT

A very fast update. At the request of the excellent Lars Brinkhoff via GitHub I have added in the language Forth to be one of the supported languages inside searchcode.

An example search which shows this working would be the following https://searchcode.com/?q=forth&loc=0&loc2=10000&lan=181

I had to solve a number of interesting problems inside searchcode to support this change. For pragmatic reasons the way searchcode identifies what language any piece of code is written in is to run it though CLOC (Count Lines Of Code). Written in perl it does a reasonably good job of pulling out metadata for any given piece of code. However since my perl ability is poor at best submitting a patch to support forth was not going to be an option.

Hacker proves with Open Data that Microsoft license costs don't matter

Welcome to Marco Fioretti's website! on Stop at Zona-M — Wed, 02 Sep 2015 14:09:07 GMT

(no, not really but…) In December 2014, italian Prime Minister Matteo Renzi annunced soldipubblici.gov.it, a Web portal that provides official Open Data on public spending (“Soldi Pubblici”, that is) in Italy. Within a few hours, an italian Hacker,

Tomcat Hates Encoded Slashes

Train of Thought — Wed, 02 Sep 2015 03:00:00 GMT

Tomcat will treat encoded slashes in a URI as path delimiters. I'll demonstrate how this can cause you problems, why it occurs, and how you can work around it.

Sierra Leone travel bookmarks

xenodium.com @alvaro — Wed, 02 Sep 2015 03:00:00 GMT

London travel bookmarks

xenodium.com @alvaro — Wed, 02 Sep 2015 03:00:00 GMT

Chiswick House & Gardens.
Heath Robinson Museum.
London Library (book your free tour).
Quaker gardens, Islington.
Soho Theatre (not tried yet).
The best brunch London 2020 | CN Traveller.
The most beautiful restaurants in London for 2020 | CN Traveller.

Tao Te Ching

june.kim — Wed, 02 Sep 2015 03:00:00 GMT

Hello World

Posts on Sean Lane — Wed, 02 Sep 2015 03:00:00 GMT

Update as of November 2018: I’ve since moved to https://sean.lane.sh, just keeping this around for sentiment’s sake.

While most definitely not the first or the last, hopefully this is a seminal moment for my own personal site. I hope to stay with seanlane.net after putting my wife through the torment of having me decide which of several domain names to settle on.

I thought sean.tl would be a nice little domain hack of my first name and initials, but it proved to be fairly confusing, as most people had no idea what the ‘.tl’ was supposed to go with. Of course, all of the sean.* domains that make more sense have been taken, and seanlane.com is devoted to a redirect to a concert scheduling site of some sort for a musician that shares both my given and last names. But I was pretty happy to find this domain, and it seems to be working just fine.

In any case, I am using Github Pages to host this with a nice little Jekyll theme from JekyllNow.com since I need this to be up and running for a class I am taking this semester. Maybe I can see if there is anything more elegant I can do with it later, but this is more than enough for the time being.

Exporting Documents from KnowledgeTree 3.7.0.2

Ben E. C. Boyter — Wed, 02 Sep 2015 01:13:04 GMT

I was recently tasked with exporting a large collection of documents from KnowledgeTree (KT) for a client. The collection was too large to use the download all functionality and too wide to attempt to export each folder individually.

I had played around with the WebDav connection that KT provides but it either didn’t work or was designed deliberately to not allow exporting of the documents.

I looked at where the documents were stored on disk but KT stores them as numbered files in numbered directories sans extension or folder information.

The 5th Wave

🦉 olshansky 🦁 — Tue, 01 Sep 2015 06:59:38 GMT

Olshansky's review of The 5th Wave

Forest

june.kim — Tue, 01 Sep 2015 03:00:00 GMT

OS X Software Update Channels For Betas

smcleod.net — Tue, 01 Sep 2015 03:00:00 GMT

Set update channel to receive developer beta update

sudo softwareupdate --set-catalog https://swscan.apple.com/content/catalogs/others/index-10.11seed-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz

Set update channel to receive public beta update

sudo softwareupdate --set-catalog https://swscan.apple.com/content/catalogs/others/index-10.11beta-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz

List available updates

sudo softwareupdate --list

Set update channel to receive default, stable updates

sudo softwareupdate --clear-catalog

Show current settings

defaults read /Library/Preferences/com.apple.SoftwareUpdate.plist

Write setting manually

defaults write /Library/Preferences/com.apple.SoftwareUpdate CatalogURL https://swscan.apple.com/content/catalogs/others/index-10.11beta-10.11-10.10-10.9-mountainlion-lion-snowleopard-leopard.merged-1.sucatalog.gz

Disabling graphical password prompts

Separate Concerns — Mon, 31 Aug 2015 22:30:00 GMT

These days Linux systems tend to open graphical password prompts when a CLI application needs user authentication. I don’t know about you but I really don’t like that.

The first offender is git, which uses x11-ssh-askpass if installed. The simplest solution would be not to install it but it is a dependency of virt-manager on Arch Linux… Thankfully you can tell git not to use it:

git config --global core.askpass ""

The second one, in my case, was pass. If you try to use it in Gnome, the keyring hijacks the GPG agent and you get that message (plus a graphical prompt):

gpg: WARNING: The GNOME keyring manager hijacked the GnuPG agent.
gpg: WARNING: GnuPG will not work properly - please configure
that tool to not interfere with the GnuPG system!

The Gnome keyring is an annoying piece of software that replaces password prompts for several tools including SSH and GPG. you can disable it this way:

mkdir -p ~/.config/autostart
cd !$
cp /etc/xdg/autostart/gnome-keyring-* .
for i in *; do echo "Hidden=true" >> $i; done

… but then GPG will use yet another graphical prompt! To finally stay in your terminal, create the file ~/.gnupg/gpg-agent.conf with the following content:

pinentry-program /usr/bin/pinentry-curses

Tomfoolery with Typography: Emphasizing things

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Mon, 31 Aug 2015 20:59:53 GMT

When writing your magnum opus, you might want to put emphasis on certain facts, names, and so on. There are three good ways and one bad way of doing this.

TL;DR: Do not use underlining for emphasis in a text. Use italics, bold, or small caps, depending on your font and your fancy.

Let us take a look at the bad way first: Underlining. Depending on the font you use, it might look very irregular. For web content, it might confuse your readers because they expect a hyperlink. Also, it is slightly hard to read.

This is a remnant from the age of typewriters where other text decorations were impossible to do. Let this remnant enjoy its well-deserved retirement. In modern times, we can do without it. It evokes, at least for me, the distinct style of yellow press publication.

So, what are the three good ways, then? To wit, italics, bold, or small caps. Italics are easily achieved in LaTeX with \textit{Text} (or \emph{Text}, which is preferable for a number of reasons; see below) or in HTML with Text. Similarly, you obtain a bold decoration with \textbf{Text} in LaTeX or Text in HTML—although  only defaults to bold text. How it is actually rendered depends on the CSS. Last, small caps are capital characters that are set similar to surrounding lower case letters, making them not as conspicuous as upper-case characters. Whether they look good or not depends very much on the font you use. In LaTeX, you can get them using \textsc{Text}, in HTML you have to add font-variant:small-caps; to the current style.

This is pretty straightforward. The one thing you should avoid is combining different modes of emphasis. Bold and italics do not mix well in most cases.

As a parting thought, some words on \emph in LaTeX: You should rather use \emph than \textit because the former command is somewhat smart and can recognize nesting. For example, if your personal style is to typeset figure captions in italics but you need to emphasize something in the caption, \emph will detect this and render the text upright. When using \textit, nothing would happen here because, well, the text is already in italics. If this does not convince you, just think about the flexibility offered by \emph: You could redefine it to work differently for some sections in your document if that strikes your fancy.

To end with a happy note for the poor underlined text: You can use underlining to mark corrections in a text. This might be especially helpful when collaborating on a document. But do not use underlines for anything that is going to be included in the final version of a document.

Links - August 31, 2015

Avy Faingezicht — Mon, 31 Aug 2015 03:00:00 GMT

A blazing fast geo database with LevelDB, Go and Geohashes

Fabrice Aneche — Mon, 31 Aug 2015 01:25:02 GMT

You probably have heard of LevelDB it’s a blazing fast key value store (as a library not a daemon), that uses Snappy compression. There is plenty of usages for it, the API is very simple at least in Go (I will be using Goleveldb). The key is a []byte the value is a []byte so you can “get”, “put” & “delete” that’s it. I needed a low memory low cpu system that could collect millions of geo data and query over them, Geohash has an interesting property you can encode longitude and latitude into a string : f2m616nn this hash represents the lat & long 46.

I have become Crazy Cat Lady, the Buyer of Cat Strollers

Liza Shulyayeva — Sun, 30 Aug 2015 12:27:41 GMT

I am sorry in advance for what you are about to read. If witnessing the evolution of a mere human into Crazy Cat Lady is something you may find disturbing, turn back now and pretend you never came to this godforsaken place.

The answers are there

june.kim — Fri, 28 Aug 2015 03:00:00 GMT

Links - August 28, 2015

Avy Faingezicht — Fri, 28 Aug 2015 03:00:00 GMT

Why you are very often wrong when you copy a full web page somewhere else online

Welcome to Marco Fioretti's website! on Stop at Zona-M — Thu, 27 Aug 2015 20:12:54 GMT

If and when the author (including me, I routinely do that on OTHER websites I run, or contribute to!) of some web page, video etc.. tells you that you can copy it all elsewhere, by all means DO it. In all other cases, including “sharing” them on Facebook or similar networks, or sending the full thing via email, you do a serious disservice… not just to that author but, **above all, **to all the people with which you “share” that stuff.

About the Stop (original version)

Welcome to Marco Fioretti's website! on Stop at Zona-M — Thu, 27 Aug 2015 18:17:53 GMT

Welcome to the Stop, a website by Marco Fioretti, author, teacher and a few other things. The sidebar on this website includes more news about my courses and other projects. The Stop hosts, in the four sections explained below, information and writings directly related to my work, including online versions of some of my publications. Some pieces about trekking and backpacking are at Strider, and anything else that I feel useful to share stays in my other “Tips” blog.

Links - August 27, 2015

Avy Faingezicht — Thu, 27 Aug 2015 03:00:00 GMT

Sampling the Visible Sphere

Fred Akalin — Wed, 26 Aug 2015 10:00:00 GMT

(Note: this article is a summary of this thread on ompf2.)

The usual method for sampling a sphere from a point outside the sphere is to calculate the angle of the cone of the visible portion and uniformly sample within that cone, as described in Shirley/Wang.

However, one detail that is glossed over is that you still need to map from the sampled direction to the point on the sphere. The usual method is to simply generate a ray from the point and the sampled direction and intersect it with the sphere. However, this intersection test may fail due to floating point inaccuracies (e.g., if the sphere is small and the distance from the point is large).

I've found a couple of existing ways to deal with this. As described in the pbrt book, pbrt simply assumes that the ray just grazes the sphere if the intersection fails, and then projects the center of the sphere onto the ray (code here). mitsuba moves the origin of the ray closer to the sphere (in fact, from within it) before doing the test (falling back to projecting the center onto the ray if that still fails) (code here).

However, this seems inelegant. I've come up with a better way, which involves converting the sampled cone angle $θ$ (as measured from the segment connecting the point to the sphere center) into an angle $α$ from the inside of the sphere, and then simply using $α$ and the sampled polar angle $\varphi$ onto the sphere. This turns out to be simple, and in my unscientific tests a bit faster.

Here's a crude diagram showing the geometry:

You can see that \[ L = d \cos θ - \sqrt{r^2 - d^2 \sin^2 θ} \] and also by the law of cosines, \[ L^2 = d^2 + r^2 - 2 d r \cos α\text{.} \] We're actually more interested in $\cos α$, so solving for that we get \[ \cos α = \frac{d}{r} \sin^2 θ + \cos θ \sqrt{1 - \frac{d^2}{r^2} \sin^2 θ}\text{.} \] An alternate form, which may be easier to analyze, recalling that $\sin θ_{\max} = r/d$, is \[ \cos α = \frac{\sin^2 θ}{\sin θ_{\max}} + \cos θ \sqrt{1 - \frac{\sin^2 θ}{\sin^2 θ_{\max}}}\text{.} \]

So sampling pseudocode would look like:

(cos θ, φ) = uniformSampleCone(rng, cos θmax)
D = 1 - d² sin² θ / r²
if D ≤ 0 {
  cos α = sin θmax
} else {
  cos α = (d/r) sin² θ + cos θ √D
}
ω = sphericalDirection(cos α, φ)
pSurface = C + r ω

I haven't done any analysis yet on the most robust way [in the floating-point sense] to do the calculations above.)

There's no backfacing since we clamp $\cos α$ to $\sin θ_{\max}$, which is analogous to the case when the ray from $P$ misses the sphere.

Note that one cannot just compute $α_{\max}$ and uniformly sample the cone from inside the sphere, as that doesn't produce the same distribution over the visible region as sampling the cone from outside the sphere. To preserve correctness, you would have to use the (uniform) PDF over the surface area of the visible portion of the sphere, but you would have to then convert that to a PDF with respect to projected solid angle from $P$, which is suboptimal to just doing the sampling with respect to projected solid angle from $P$ as described above.

Like this post? Subscribe to my feed or follow me on Twitter .

Questions on Success

june.kim — Wed, 26 Aug 2015 03:00:00 GMT

Links - August 26, 2015

Avy Faingezicht — Wed, 26 Aug 2015 03:00:00 GMT

Decoding CAPTCHA's Handbook

Ben E. C. Boyter — Wed, 26 Aug 2015 01:49:31 GMT

Some time ago I wrote an article about Decoding CAPTCHA’s which has become what appears to be the first resource most people encounter when searching for information in the decoding CAPTCHA space.

I had continued to write about CAPTCHA’s over the years with posts scattered around the web. A while ago I started to consolidate all of my content on this blog and realised that I had considerably more CAPTCHA related articles then I thought. Some were in an unfinished or unpublished state. I had considered posting them all online but instead decided to polish it all up into a much better resource and publish it as a book.

Zoolander

🦉 olshansky 🦁 — Tue, 25 Aug 2015 18:40:33 GMT

Olshansky's review of Zoolander

Straight Outta Compton

🦉 olshansky 🦁 — Tue, 25 Aug 2015 18:36:13 GMT

Olshansky's review of Straight Outta Compton

A Heed to the Salaryman

june.kim — Tue, 25 Aug 2015 03:00:00 GMT

Links - August 25, 2015

Avy Faingezicht — Tue, 25 Aug 2015 03:00:00 GMT

The Fall of Hyperion (Hyperion Cantos, #2)

🦉 olshansky 🦁 — Tue, 25 Aug 2015 03:00:00 GMT

Olshansky's review of The Fall of Hyperion (Hyperion Cantos, #2) by Dan Simmons

Zero Downtime Deployment with AWS ECS and ELB

Brightball Articles — Tue, 25 Aug 2015 03:00:00 GMT

As development teams push farther toward continuous delivery, deploying updates to an application without disruption to users is constantly becoming a more sought-after practice. Amazon’s EC2 Container Service helps to make that easier than ever with tight Elastic Load Balancer integration.

Custom endpoint handlers with Eve

Nicola Iarocci — Mon, 24 Aug 2015 03:00:00 GMT

On Stack Overflow and the Eve mailing list, but also in my mailbox and even on Twitter, I get a lot of enquiries on how to build custom endpoints within a Eve-powered RESTful application. Now, since within Eve all endpoints are fully customizable, what they really mean is:

How do I setup endpoints without any binding to a data entity, just connected to a custom method?

They would like to call something like /mycustomendpoint and get the response from a method they have defined somewhere in the Python sources. The standard endpoint-entity mapping provided by Eve covers 90% of their needs, but sometimes they just need endpoints that have nothing to do with data entities.

Links - August 24, 2015

Avy Faingezicht — Mon, 24 Aug 2015 03:00:00 GMT

Use ImageMagick to auto-orient images

xenodium.com @alvaro — Sun, 23 Aug 2015 03:00:00 GMT

Recently needed to rotate images based on EXIF metadata. ImageMagick to the rescue:

$ for i in *.png; do convert -auto-orient "$i" "$i"; done

Bengali Macher Jhol

xenodium.com @alvaro — Sun, 23 Aug 2015 03:00:00 GMT

My friend Sakhya brought me the wonderful Cookbook of Regional Cuisines of India. After improvisations and substitutions, here's my attempt at making Bengali Machcher Jhol:

Running Laravel's Lumen On Google App Engine

KernelCurry — Sun, 23 Aug 2015 03:00:00 GMT

When looking for a more stable hosting solution for my Magic: The Gathering Api, I stumbled across Google’s App Engine. If you have not read up on this solution, I would recommend it.

To get Laravel’s Lumen running correctly on Google App Engine, there are a few questions that we need to answer. Lets take these one at a time.

How does the application know it is running on Google App Engine?

One of the easiest ways to figure out if the application is running on Google App Engine is to check if the .env file is present. This is because the deployment “ignore” RegEx includes any files that start with a . (AKA Unix hidden files). This function will come in handy when answering the next questions.

Hello

Thought Eddies — Sun, 23 Aug 2015 00:59:48 GMT

Hello

New habits for 2015

xenodium.com @alvaro — Sat, 22 Aug 2015 03:00:00 GMT

20 min morning meditations.
A better way to tie your shoes.
Cold showers (all of them!).
Keys, wallet, phone, badge, and headphones live together.
Listen to audio books.
Morning runs.
Nightly flossing.

Meditation retreats bookmarks

xenodium.com @alvaro — Sat, 22 Aug 2015 03:00:00 GMT

Mindfulness/meditation bookmarks

xenodium.com @alvaro — Sat, 22 Aug 2015 03:00:00 GMT

Learning bookmarks

xenodium.com @alvaro — Sat, 22 Aug 2015 03:00:00 GMT

Bundi travel bookmarks

xenodium.com @alvaro — Sat, 22 Aug 2015 03:00:00 GMT

Bundi Haveli (accomodation).
Hadoti Palace (accomodation).
Haveli Braj Bhushan Ji ki (accomodation).
Haveli Katkoun Guest House (accomodation).
Kasera Paradise (accomodation).

Unpopular

june.kim — Fri, 21 Aug 2015 03:00:00 GMT

It's time for Open Data in and from (not "ABOUT"!) schools

Welcome to Marco Fioretti's website! on Stop at Zona-M — Thu, 20 Aug 2015 12:19:58 GMT

Preface

This essay expands a proposal on Open Data in schools that I made in 2011, which requires very little, if any, funding and central authorization/coordination to be implemented. As of this writing, I know of no other proposal of the same kind, with the exception of this 2012 presentation from New Zealand. Also, I have not heard of any large scale implementation, or had occasion to do any real work on this topic. However, I am even more convinced now than in 2011 that the idea has a great potential.

Smart dust: coming soon to a lung near you?

Welcome to Marco Fioretti's website! on Stop at Zona-M — Thu, 20 Aug 2015 11:21:04 GMT

You really can’t make this stuff up: I have just found, one after another in my RSS feed, two unrelated articles on Smart Dust that you must really read side by side: one about a wonderful future, another unintentionally exposing some of its, apparently, not-yet-considered consequences.

Self identity

june.kim — Thu, 20 Aug 2015 03:00:00 GMT

Links - August 20, 2015

Avy Faingezicht — Thu, 20 Aug 2015 03:00:00 GMT

Snappy Scope: Progress Update

kyrofa's blog — Thu, 20 Aug 2015 03:00:00 GMT

I’ve previously written about Snappy and Unity 8’s Snappy Scope, back when I was first getting started on it. Since then, both the scope and Ubuntu Core itself have come a long way, so I thought it was time for an update and a quick demo. When I was working on Snappy back when I wrote the other post I was running Ubuntu Core virtually, since I was using my embedded devices for other tasks.

Carved Pen

june.kim — Wed, 19 Aug 2015 03:00:00 GMT

Links - August 19, 2015

Avy Faingezicht — Wed, 19 Aug 2015 03:00:00 GMT

Strategic Game of Life: Mobile Edition

Thomas Hunter II — Wed, 19 Aug 2015 03:00:00 GMT

Tomfoolery with Typography: Dots and dashes

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Tue, 18 Aug 2015 22:39:28 GMT

For the first “real” episode in this series, I wanted to start very slowly. Let us take a look at very basic typographic symbols, namely dots and dashes (and some spaces). Except for the rules about hyphens, dashes, and the like, all the rules presented here are more about aesthetics and custom.

Dots

We use a simple full stop (or period for you Americans) to indicate the end of a sentence. Nothing new here so far.

When we want to indicate a multiplication in a formula, sometimes it makes sense to show a multiplication symbol. In this case, the proper symbol to use is \cdot (for LaTeX users) or ~~·~~⋅ (for HTML). This symbol is typeset as ⋅ and defines a binary operator. A similar symbol with a different semantic intent is · (HTML) or \textperiodcentered (LaTeX); rendered as ·, it can be used as to indicate a transition between words, for example on a business card. A sample business card might read “John Doe · Tinker & Tailor”.

Never, never use \times or × to indicate multiplication in any mathematics text written for university students. In the classroom, you might find “5 × 4 = 20”, but the × symbol is customarily used to indicate things like a cross product in mathematics.

If you want to indicate that something is missing from a quotation, a typeset matrix, or anything else, use the wonderful ellipsis character. It consists of three dots that are spaced differently than three regular dots. In LaTeX, you can use \dots. In HTML, use …. It looks like this:

1,2,3,…,n

The advantage of the predefined ellipsis character is that it will not be broken up. Some style guides ask you to write an ellipsis as a sequence of dots and spaces. They are wrong. There is a difference between “. . .” and “…” in all fonts.

Dashes

Now let us dash to the dashes! There are three types of dashes. First, we have the simple hyphen, which is technically not a dash but I am calling it one anyway because one usually thinks of it as a dash. Second, we have the en-dash. Third, we have the em-dash.

The hyphen is typeset using a simple - character. We use it to indicate a break in a word, when said word needs to be split because the end of a line has been reached. We also use it to spell some composite words, such as good-hearted or mother-in-law. A further use of the hyphen is to remove ambiguities in some compound adjectives. For example, small-arms fire, high-school students, and so on. XKCD 37 also has a nice example for this.

I do not know all the rules about when to use a hyphen or not. As a non-native speaker, I am slightly biased here. When in doubt, consult your dictionary of choice. At least you now know what the hyphen is meant for.

Moving on to the next dash, the en-dash. In LaTeX we set it by typing two dashes (--). In HTML, we write – to obtain “–”. The en-dash is customarily used to indicate ranges. For example:

The theorem is proved on pages 23–42.

1887–1895 was the period of the first construction of the Kiel Canal.

Most bibliographical services in the web are unable to get the en-dash right. I am looking at you, ACM and IEEE.

The en-dash is also used to indicate joint authorship, such as the Barnes–Hut simulation.

Last, the em-dash. It is the longest of all the dashes. In LaTeX, it is typeset by three hyphens in a row ---. In HTML, use the entity —. The em-dash is rather long, “—”. It is used to indicate a break between parts of a sentence. Most typographers consider it stronger than a comma but weaker than a semicolon or a pair of parentheses. If this comparison does make sense to you, please use the em-dash. Here is an example from my own writing:

Topologists aim to identify invariants of such spaces—properties that do not change when the space is stretched, bent, and twisted by homeomorphisms.

I typically use the em-dash without any further spaces because I consider it long enough to indicate a break by its own. This again is a matter of personal style. If you want to use dash to indicate pauses, omissions, and so on, please use the em-dash. It gives the eye something to follow and indicates a break. To bore you with a personal anecdote: When converting some eBooks to my Kindle, em-dashes are made into hyphens. This really confuses me when reading because I expect a compound word or a line-break but get a parenthetical remark.

In short: Dots. Dashes. Use them properly, please.

Update (2026-01-12): Thanks to Callum, I learned about ⋅ and that I was wrong about how to use ·. I have updated the article accordingly. I love receiving feedback like this! 🤩

Running three hours of Ruby tests in under three minutes

Ben E. C. Boyter — Tue, 18 Aug 2015 11:18:47 GMT

Recently the very cool hard working developers working on Stripe released a post about how they modified their build/test pipeline to reduce their test suite runtime from 3 hours to about 3 minutes.

The article is very much worth reading, as is the discussions that have come around it including those on Hacker News.

A few key takeaways,

For dynamic languages such as Ruby or Python consider forking to run tests in parallel

Thanking Oh-My-Vagrant contributors for version 1.0.0

The Technical Blog of James on purpleidea.com — Tue, 18 Aug 2015 06:12:26 GMT

The Oh-My-Vagrant project became public about one year ago and at the time it was more of a fancy template than a robust project, but 188 commits (and counting) later, it has gotten surprisingly useful and mature.

james@computer:~/code/oh-my-vagrant$ git rev-list HEAD --count
188
james@computer:~/code/oh-my-vagrant$ git log $(git log --pretty=format:%H|tail -1)
commit 4faa6c89cce01c62130ef5a6d5fa0fff833da371
Author: James Shubin <james@shubin.ca>
Date:   Thu Aug 28 01:08:03 2014 -0400

    Initial commit of vagrant-puppet-docker-template...
    
    This is an attempt to prototype a default environment for
    vagrant+puppet+docker hacking. More improvements are needed for it to be
    useful, but it's probably already useful as a reference for now.

It would be easy to take most of the credit for taking the project this far, as I’ve been responsible for about 87% of the commits, but as is common, the numbers don’t tell the whole story. It is also a bug (but hopefully just an artifact) that I’ve had such a large percentage of commits. It’s quite common for a new project to start this way, but for Free Software to succeed long-term, it’s essential that the users become the contributors. Let’s try to change that going forward.

Reading Technical Documentation

june.kim — Tue, 18 Aug 2015 03:00:00 GMT

Links - August 18, 2015

Avy Faingezicht — Tue, 18 Aug 2015 03:00:00 GMT

A Journey to the Centre of the Earth

Apurva Shukla's RSS Feed — Tue, 18 Aug 2015 02:39:15 GMT

⭐ ⭐ ⭐ Written in a time when scientific exploration was pushing the bounds of reality, Jules Verne conjures up the unfathomable of ideas…

Why I write in the morning

june.kim — Mon, 17 Aug 2015 03:00:00 GMT

Links - August 17, 2015

Avy Faingezicht — Mon, 17 Aug 2015 03:00:00 GMT

Tomfoolery with Typography: Introduction

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Sun, 16 Aug 2015 15:12:15 GMT

Inspired by Sheldon’s “Fun with Flags” series, I decided to collect my thoughts about the beautiful subject of typography in a similar way. My friends and students know that I can go on and on about this topic. Maybe writing it down like this decreases the length of my rants in the real-world.

For this first part, I want to explain some definitions and dispel some myths about typography.

What is typography?

Typography is the art and craft of arranging type to make written language readable and appealing.

In this definition, type refers to the composition of text by some means. I am most used to computer typesetting, so I am assuming that we are doing electronic typesetting here. Most of my examples will either refer to HTML or LaTeX, because I am most familiar with them.

Note that the definition contains the words art and craft. Typography is thus decidedly not all about making a document look “nice” or anything. Some of the rules are grounded in the reality of the perception of readers—see below for some explanations.

What makes you qualified to write about this?

To be honest, nothing specifically. I like good typography. Over the course of my career as a student and a researcher, I have always given some thought about presenting my text in an aesthetically pleasing manner.

I am not claiming that my way of doing things is the best way. I am also not claiming that my documents are the best-looking ones out there. Furthermore, I am well aware of the fact that typography for web pages leaves some things to be desired. For example, I am not quite satisfied with the way quotes or apostrophes are typeset. But this is a limitation of the web medium, not a principal limitation.

Are you saying that looks are more important than content?

No. Typography is not about making bad content look good. You are confusing this with marketing. Of course, your content needs to be worth the effort of typesetting. The problem with electronic typesetting is that—whether we want to or not—each and everyone of us has become a little typographer. Every word processing software permits a myriad of ways of adjusting the look of text. So, unless you are writing everything in a text editor and only apply the design later on—which, coincidentally, might not be the worst idea—you should at least be aware of the basic principles of typography.

But this is all subjective!

As a matter of fact, it is not. At least not entirely. During the course of this series I also want to provide pointers to relevant research whenever applicable. I plan on separating the purely aesthetic aspects, which are certainly subjective or reliant on a fashion-du-jour, from the aspects that have a solid basis in perception.

Let us start with this right now. Here are some interesting research findings about typography:

In Reader Preferences and Typography, Tinker and Paterson reported that there is a close agreement between the apparent legibility and the apparent pleasingness of a text. The study is slightly scant on details, though, as Tinker and Paterson only compared text set in lower case with text set in bold face or all capitals. Still, it seems to point in the direction that certain text markups shall only be used sparingly. Bold face, for example, draws the attention of a reader towards a single word or phrase. The effect is lost if everything is set in bold face.
In Typography and Readability, Burtt examined the readability of text if certain typographical elements are changed. The selection of the font face proved to be somewhat significant, for example. Very ornate fonts, such as Cloister Black were deemed to confuse some readers. Of course, this is not entirely surprising, but it hints at the importance of selecting the “right” font. Furthermore, Burtt reported that combining several font faces can actually decrease the reading efficiency by as much as 11%. Eye movement observations also suggested that long texts should not consist exclusively of capital letters. Burtt also observed that line lengths should not be too large. For 10 point type, line length should vary between 75mm and 90mm.
In The Effects of Line Length on Children and Adults’ Online Reading Performance, Bernard et al. found that medium (65–75 characters per line) or narrow length texts (45 characters per line) are preferred for online reading by adults and children, respectively.
In The Aesthetics of Reading, Larson and Picard examined the performance of participants in cognitive tasks after exposing them to examples of bad and good typography. While Larson and Picard concluded that most readers do not perceive very intricate details such as ligatures or small caps (which may thus be thought of belonging to the aesthetics part of typography), they found that readers greatly prefer good typography to bad typography. I would take the cognitive differences with a grain of salt, though, because the sample size does not seem to be large enough—only 20 subjects, which were divided into two groups. Yet, subjects rated the text with good typography to be significantly easier to read.

I hope I have convinced you that some typographical standards are worth thinking about. In the forthcoming posts, I will always refer to relevant research if applicable, or explicitly state that a rule is “only” based on aesthetic preferences.

Stay tuned.

Links - August 16, 2015

Avy Faingezicht — Sun, 16 Aug 2015 03:00:00 GMT

The Age of Adaline

🦉 olshansky 🦁 — Sat, 15 Aug 2015 18:25:12 GMT

Olshansky's review of The Age of Adaline

Before Midnight

🦉 olshansky 🦁 — Sat, 15 Aug 2015 16:07:00 GMT

Olshansky's review of Before Midnight

Writing Specifically

june.kim — Sat, 15 Aug 2015 03:00:00 GMT

Links - August 15, 2015

Avy Faingezicht — Sat, 15 Aug 2015 03:00:00 GMT

The Four Character Group Archetype

Tilting at Windmills — Sat, 15 Aug 2015 03:00:00 GMT

I've noticed that when a work of fiction focuses on a group of four characters, the roles those characters play within the group fall into a…

The Flash: Season 1

🦉 olshansky 🦁 — Sat, 15 Aug 2015 02:20:26 GMT

Olshansky's review of The Flash: Season 1

South Park: Season 1

🦉 olshansky 🦁 — Sat, 15 Aug 2015 02:19:47 GMT

Olshansky's review of South Park: Season 1

Community: Season 1

🦉 olshansky 🦁 — Sat, 15 Aug 2015 02:19:03 GMT

Olshansky's review of Community: Season 1

Modern Family: Season 1

🦉 olshansky 🦁 — Sat, 15 Aug 2015 02:18:39 GMT

Olshansky's review of Modern Family: Season 1

American Dad!: Season 1

🦉 olshansky 🦁 — Sat, 15 Aug 2015 02:18:18 GMT

Olshansky's review of American Dad!: Season 1

Family Guy: Season 1

🦉 olshansky 🦁 — Sat, 15 Aug 2015 02:18:01 GMT

Olshansky's review of Family Guy: Season 1

The Big Bang Theory: Season 1

🦉 olshansky 🦁 — Sat, 15 Aug 2015 02:17:46 GMT

Olshansky's review of The Big Bang Theory: Season 1

Upgrading PL30 headphones

xenodium.com @alvaro — Fri, 14 Aug 2015 03:00:00 GMT

I've loved my Soundmagic PL30 in-ear headphones. They're relatively inexpensive, comfortable, and great for exercising (they stay in). Audio quality and bass have been good enough (I don't need much). Unfortunately, I've had two pairs of PL30's and both stopped working after a year or two. I'm replacing the last pair with RHA's MA750 (an upgrade, me hopes).

Other contenders considered: Etymotic Research HF5, and Shure SE215. Also considered bluetooth alternatives like JayBird BlueBuds X and Plantronics BackBeat GO 2.

I'm somewhat nervous to pay more for a pair of headphones. Let's hope they don't meet the same unfortunate fate. We'll see.

The Phantom Tollbooth

june.kim — Fri, 14 Aug 2015 03:00:00 GMT

Entity Framework Code First type safe eager loading

English articles on AlesRuzicka.net — Fri, 14 Aug 2015 03:00:00 GMT

In Entity Framework Code First, you can achieve eager loading of entities by using Include method on DbQuery in from DbContext of your application.

One line password generation in .Net using LINQ

English articles on AlesRuzicka.net — Fri, 14 Aug 2015 03:00:00 GMT

When I wrote this code in use-once application, I thought that is rather weird part of code, but works just fine for this once case.
Then I realized, there is nothing wrong with the code. People, who are familiar with LINQ will have little problem reading it and it will work 100% reliably.

Quotes

xenodium.com @alvaro — Thu, 13 Aug 2015 03:00:00 GMT

"Being good at something is about being curious enough to explore things to a level where most people give up."
"The world is a book and those who do not travel read only one page." - Augustine of Hippo.
"National identity is not your only identity." - Xiaolu Guo?
"Choose your words carefully. Words are cheap, but their effect can be expensive."

Bhutan travel bookmarks

xenodium.com @alvaro — Thu, 13 Aug 2015 03:00:00 GMT

Taktsang (Tiger’s Nest) Monastery.

Leather Field Notes Cover

june.kim — Thu, 13 Aug 2015 03:00:00 GMT

Tools I could not live without 2015

English articles on AlesRuzicka.net — Thu, 13 Aug 2015 03:00:00 GMT

I try not to overdo it with amount of tools I use, but I’ll make a short list of tools I use right now.

Groundhog Day

🦉 olshansky 🦁 — Thu, 13 Aug 2015 02:41:13 GMT

Olshansky's review of Groundhog Day

Deep Neural Networks with Caffe

Umar Nizamani | RSS Feed — Wed, 12 Aug 2015 21:08:11 GMT

Working in experiential marketing, I wanted to be able to target campaigns more intelligently than conventional wisdom and use technology to…

Synchronous multiplexing for sockets (starring C++11)

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Wed, 12 Aug 2015 19:58:45 GMT

In my previous post on network sockets, I described a simple implementation of a server and a client socket in C++11. The initial version was only capable of reacting to a new client connection. What about a socket being ready for reading, though?

`select()` and socket sets

There are numerous ways of doing multiplexing for sockets—of course I used the easiest one which is the worst and slowest. Anyway, after some quality time with the socket documentation, I implemented the following procedure:

Create two empty socket sets, a master and a client socket. The sockets API refers to them as fd_set.
Create a server socket (as in the previous post) and add it to the master set.
Loop forever! Copy the master socket set to the client socket set and call select().
If the call to select() finishes without an error, one of the sockets in the set is ready for something. This something might be a read operation or a new client.
Handle this by creating new client sockets as appropriate, calling the corresponding callback functions using std::async, and updating the socket sets.

Sounds rather straightforward, but the corresponding code is messy and not yet very smart. For example, I am looping over all possible file descriptors and I had to adjust the ClientSocket class to un-register at the server class once a socket is closed. I am not quite content with the implementation.

How to use it?

The API is still using the callback approach introduced in the previous post. The cool thing is that this now also works whenever a client socket has something to say. For example, to implement RFC 862, the Echo protocol, we merely need the following code:

server.onRead( [&] ( std::weak_ptr<ClientSocket> socket )
{
  if( auto s = socket.lock() )
  {
    auto data = s->read();
    s->write( data );
  }
} );

Note that I have changed the std::unique_ptr to an std::weak_ptr because the server is now responsible for managing the client sockets. Since all handlers are called being asynchronously, it is possible that a socket is already unavailable because it has been closed.

Where is the code?

I have updated the GitHub repository introduced in the last post. The repository now also contains a simple echo server implementation.

The code is released under an MIT licence.

No Country for Old Men

🦉 olshansky 🦁 — Wed, 12 Aug 2015 11:40:05 GMT

Olshansky's review of No Country for Old Men

Frequencies

🦉 olshansky 🦁 — Wed, 12 Aug 2015 11:29:41 GMT

Olshansky's review of Frequencies

Cooking bookmarks

xenodium.com @alvaro — Wed, 12 Aug 2015 03:00:00 GMT

Academy Of Management

june.kim — Wed, 12 Aug 2015 03:00:00 GMT

Links - August 12, 2015

Avy Faingezicht — Wed, 12 Aug 2015 03:00:00 GMT

Back To XP

Craig Pardey — Wed, 12 Aug 2015 03:00:00 GMT

Agile has gained a lot of traction in large enterprises in recent years but I’m concerned about the specifics of how teams are implementing it. I’ve seen everything from traditional eXtreme Programming (XP) practices to Scrum, but nothing scares me more than the teams that self-identify as “agile” but do nothing more than hold a daily team meeting.

Agile means different things to different people, but it doesn’t matter which methodology is used as long as the principles outlined in the Agile Manifesto are adhered to. Holding a daily hour-long meeting does not an agile team make. It’s almost anti-agile.

How Haskell models IO

λm.me — Wed, 12 Aug 2015 03:00:00 GMT

Edit 2: I've written two, better articles about IO in Haskell, see the my haskell-study-plan and my book

Edit: After writing this post I turned to reddit for advice on how to make this post better, and even after a complete re-write It still felt lacking. After asking about it at #haskell, merijn linked to this article which in my opinion explains this subject much better than mine. So you might want to read it before this post or even instead of.

Haskell is a purely functional language, but what does being "pure" mean?

The pure in purely functional means that Haskell enforces the separation between evaluating an expression and executing an expression.

An expression can be thought about simply as a value. In order to produce a value, an expression needs to be evaluated (or calculated).

While evaluated, expressions are stateless and cannot do anything other than compute a value and return it, and will also return the same value. Always. This also means it can't mutate a variable, read from file or write to standard output. We say that evaluating expressions is pure.

All expressions in Haskell are pure, which mean that we can evaluate any expression. But some expressions may need to do more than just being evaluated in order to do something useful. Some expressions needs to do other things in order to produce a meaningful value, this kind of expressions can be executed. Let's call this kind of expressions IO actions.

Producing value just by evaluating expressions is simpler and more straightforward in Haskell, while executing expressions requires more attention and can only happen at certain places. Most expressions in Haskell do not need to be executed in order to produce a meaningful value.

Haskell enforces this separation between pure expressions and IO actions using it's type system at compile time.

We can differentiate between pure expressions and IO actions by looking at the type of an expression. An IO action's type signature is slightly different from a pure function or value.

For example:

val1 :: Int

val2 :: IO Int

func1 :: Int -> Int

func2 :: Int -> IO Int

val1 and func1 are pure - they don't need to be executed to produce an Int, only evaluated. val2 and func2 are IO actions - in order to produce an Int they need to be executed. We can tell because they return an IO _ type.

Let's say we want to write a program that reads an Int from a user, and multiply it by a multiplier. Here is a naive attempt where we treat an IO Int as equal to Int, which means that when evaluating IO Int we also execute it and produce an Int:

readIntFromUser :: IO Int
readIntFromUser = ...

mul :: Int -> Int -> Int
mul = (*)

userInt :: IO Int
userInt = readIntFromUser

multiplier :: Int
multiplier = 3

main = print (mul multiplier userInt)

Well, this might be a little bit problematic, because now we lost the ability to separate evaluation from execution. Also, since the type of mul is Int -> Int -> Int, the type of mul multiplier userInt is also an Int, so now we don't know if we are going to get the same value every time or not. So we can't replace the implementation of mul by something that, for example, sums a multiplier of Ints.

Pure expressions allow us to do "programming algebra" like this without unexpected side-effects.

So, we will not treat an IO Int as an Int and thus retain the separation of evaluation and execution! (Means this is not valid Haskell code.)

Okay, so, how do we multiple a user read Int by multiplier?

Hmm. Perhaps we can use a function that can execute an IO action, thus converting IO Int to Int? Let's call a function like that execute.

But now, it is also entirely possible to rewrite multiplier like this:

multiplier :: Int
multiplier = if execute userInt == 3 then 3 else 3

multiplier, in this case, still equals 3 but now it also reads user input, so it is not pure even though it's type is Int and will return 3 every time!

So, converting an IO a type to a by executing it whenever we want is rejected. (Not valid Haskell either.)

Let's look at this from a different perspective, what if we could take the function we want to apply to the IO Int value, apply it to the Int that would be calculated when executed, and return a new IO Int with the new value after applying the function? That way we can still keep the separation of pure expressions and IO actions with types!

And indeed, we can. Using fmap.

fmap :: (a -> b) -> IO a -> IO b
fmap = ...

main :: IO ()
main = print (fmap (mul multiplier) userInt) -- ==> error: print cannot take IO Int.

Yes! We produced an IO action that will take a user input and will multiply it by multiplier! But now we have a different problem, print doesn't want to print our IO Int, maybe it wants an Int? Let's try the same trick we just discovered on print as well.

main :: IO ()
main = fmap (print (fmap (mul multiplier) userInt)) -- ==> error: main type mismatch between IO () and IO (IO ())

Hmm, we were able to apply print to our computation, but print returns an IO (), so now after using fmap our main function has a type of IO (IO ()) and not IO ().

As we said, IO actions are just like regular values that can be evaluated (in the same way a function can be evaluated). They can be returned from a function or stored in a data structure. It is just that if we want to produce a value from them, we have to execute them.

Another analogy that might be helpful is to think about IO actions as "plans" to produce a value. For example, an IO a action is a plan to produce a value of type a. The plan itself is a regular, first class value. But in order to produce the a value of the plan, we need to execute it.

Sometimes we want to return an IO action to be executed later, but in this case, we just want to execute this IO action. So, what if we could join two IO together to one IO so we can think of them as one plan to execute a value?

Apparently, we can. Using join.

join :: IO (IO a) -> IO a
join = ...

main :: IO ()
main = join (fmap (print (fmap (mul multiplier) userInt)))

Great! Now everything typechecks! But it looks like a lot of work. Can't we make it more concise?

bind :: (a -> IO b) -> IO a -> IO b
bind f x = join (fmap f x)

main :: IO ()
main = bind (print . mul multiplier) userInt

Now, what if, for example, We want to test this function by supplying a value (like -3), we can use pure to create a "plan" to produce a specific Int. By the way, this IO action will not do anything beyond returning a value when executed, since we gave it a value to produce, it doesn't need to do anything in order to produce it.

userInt :: IO Int
userInt = pure (-3)

main :: IO ()
main = bind (print . mul multiplier) userInt

Okay, But we still haven't figured out when to execute an IO action and how.

Let's decide that we execute main, by executing the IO actions that compose it, be it one IO action or more that are composed with bind, and in the order they are sequenced (since you can't print without knowing what is the value the user entered). That way, we can still execute IO actions as much as we like, but we will also be able to say "anything beyond this is pure".

A short summary

Let's stop for a moment and go over how we wanted to create and enforce the separation of pure code and IO actions and what we discovered in the process.

We decided to use types to differentiate between expressions that can be executed and expressions that cannot. Expressions that can be executed has IO in their type.
We decided that we don't want to treat IO a as a - we don't want execution to happen on evaluation.
We decided that we don't want to be able to convert IO a to a (which means to execute it anywhere we want), and so once we are in IO context, we can't "escape" it.
We decided that we can convert a to IO a using pure, supplying a value to return on execution. (you can also use a function called return that does the same thing. In GHC versions older than 7.10 and other Haskell implementations, pure, which can be found in the module Control.Applicative, is not exported by default, but return is.)
We saw that we can use pure functions like (a -> b) to change IO a to IO b, using fmap.
We saw that we can chain IO actions using bind. (Which in Haskell is known as =<<. Actually, the function >>= is more commonly used which is just =<< with the arguments flipped. With >>= it looks like we are chaining IO actions from left to right)
We decided that main will be executed by executing the IO actions that compose it.

Now, let's write the program we wanted again using what we have learned, and let's make the use of >>= a little bit more explicit by using lambda expressions instead of just currying.


readIntFromUser :: IO Int
readIntFromUser = getLine >>= (\line -> pure (read x)) -- which is the same as: getLine >>= pure . read

userInt = IO Int
userInt = readIntFromUser

multiplier :: Int
multiplier = 3

mul :: Int -> Int -> Int
mul = (*)

main :: IO ()
main = userInt >>= (\input -> pure (mul multiplier input)) >>= (\result -> print result)

Now, let's say we want to first print the user's input and then the result, we could change main to:


main :: IO ()
main = userInt >>= (\input -> pure (mul multiplier input) >>= (\result -> print input >>= (\_ -> print result)))

But this becomes a little clumsy and not very readable. Fortunately, Haskell has special syntax known as do notation, let's rewrite main using it.


main :: IO ()
main = do
  input  <- userInt
  result <- pure (mul multiplier input)
  _      <- print input
  print result

This looks better. Note that input <- userInput is analogous to userInput >>= \input -> .... Also, We can do even better than that syntax-wise! We can replace _ <- ... with just ... and we can replace result <- pure ... with let result = .... Let's see what this looks like!


main :: IO ()
main = do
  input <- userInt
  let result = mul multiplier input
  print input
  print result

Now we can write IO code to be executed, in a concise way, while still retaining the ability to separate evaluation from execution.

There are still more things you can do with IO that are more concise, but with knowing what you know now you have the full power IO and you can do whatever you want with it, including building your own higher order functions to manipulate it.

Futurists

Venam's Blog — Patrick Louis (Lebanon) — Wed, 12 Aug 2015 00:00:00 GMT

Hello fellow readers, In this post I'll introduce futurists, some people with a very special occupation.

Vagrant and Oh-My-Vagrant on RHEL7

The Technical Blog of James on purpleidea.com — Tue, 11 Aug 2015 18:09:16 GMT

My employer keeps paying me, which I appreciate, so it’s good to spend some time to make sure RHEL7 customers get a great developer experience! So here’s how to make vagrant, vagrant-libvirt and Oh-My-Vagrant work on RHEL 7+. The same steps should work for CentOS 7+.

I’ll first paste the commands you need to run, and then I’ll explain what’s happening for those that are interested:

# run these commands, and then get hacking!
# this requires the rhel-7-server-optional-rpms repo enabled
sudo subscription-manager repos --enable rhel-7-server-optional-rpms
sudo yum install -y gcc ruby-devel libvirt-devel libvirt qemu-kvm
sudo systemctl start libvirtd.service
wget https://dl.bintray.com/mitchellh/vagrant/vagrant_1.7.4_x86_64.rpm
sudo yum install -y vagrant_1.7.4_x86_64.rpm
vagrant plugin install vagrant-libvirt
wget https://copr.fedoraproject.org/coprs/purpleidea/vagrant-libvirt/repo/epel-7/purpleidea-vagrant-libvirt-epel-7.repo
sudo cp -a purpleidea-vagrant-libvirt-epel-7.repo /etc/yum.repos.d/
sudo yum install -y vagrant-libvirt    # noop plugin for oh-my-vagrant dependency
wget https://copr.fedoraproject.org/coprs/purpleidea/oh-my-vagrant/repo/epel-7/purpleidea-oh-my-vagrant-epel-7.repo
sudo cp -a purpleidea-oh-my-vagrant-epel-7.repo /etc/yum.repos.d/
sudo yum install -y oh-my-vagrant
. /etc/profile.d/oh-my-vagrant.sh # logout/login or source

Let’s go through it line by line.

Making an empty RPM

The Technical Blog of James on purpleidea.com — Tue, 11 Aug 2015 16:25:58 GMT

I am definitely not an RPM expert, in fact, I’m afraid of it, but with recent tools such as COPR, and my glorious Makefile, some aspects of it have become palatable. This post will be about a recent journey I had building the most useless RPM ever.

A video of my journey building this RPM.

Because of reasons, I wanted to satisfy an RPM dependency for a package that I wanted to install without rebuilding that RPM. As a result, I wanted to build as small an RPM as possible. This took me down a much longer path than I thought it would.

The Waste Lands (The Dark Tower, #3)

Apurva Shukla's RSS Feed — Tue, 11 Aug 2015 10:00:00 GMT

⭐ ⭐ ⭐ ⭐ Whilst the story has been slowly progressing towards the climax of finding the Dark Tower, it has certainly not lost it’s beauty! 4…

9 week half-marathon training

xenodium.com @alvaro — Tue, 11 Aug 2015 03:00:00 GMT

While reading Zen Habits: Mastering the Art of Change, I comitted to running half marathon in mid-October. That's roughly two months from now. Here's a 9 week training schedule:

WEEK MON TUE WED THU FRI SAT SUN

1 Rest 5 Km 5 Km Cycle Rest 5 Km 7 Km 2 Rest 5 Km 5 Km Cycle Rest 5 Km 8 Km 3 Rest 7 Km 5 Km Cycle Rest 5 Km 10 Km 4 Rest 8 Km 5 Km Cycle Rest 5 Km 12 Km 5 Rest 8 Km Rest 8 Km Rest 5 Km 14 Km 6 Rest 8 Km Rest 8 Km Rest 6 Km 16 Km 7 Rest 8 Km 8 Km 8 Km Rest 8 Km 19 Km 8 Rest 8 Km Rest 12 Km Rest 8 Km 16 Km 9 Rest 8 Km Rest 5 Km 5 Km Rest Race

My times:

WEEK MON TUE WED THU FRI SAT SUN

1 Rest ✘ 29:04 ✔ Rest 26:36 38:40 2 Rest 29:11 28:50 ✔ Rest 27:07 44:55 3 Rest 40:46 26:29 ✔ Rest ✘ 57:01 4 Rest 46:46 ✘ ✘ Rest 30:08 1:12:10 5 Rest 46:59 Rest 44:46 Rest 24:50 1:25:24 6 Rest 50:02 Rest 46:24 Rest ✘ 1:37:39 7 Rest 46:54 46:41 46:42 Rest ✘ 1:57:57 8 Rest 45:28 Rest 48:13 (8km) Rest 43:56 ✘ 9 Rest 44:24 Rest 27:12 26:09 Rest 1:58:28

Shanghai travel bookmarks

xenodium.com @alvaro — Tue, 11 Aug 2015 03:00:00 GMT

36 Hours in Shanghai.
Shanghai Xiaolongbao at Dumpling House Edison (on Rt 27).
Tianzi Fang street art (Google maps).
Tianzi Fang street art.
Town God's Temple, street Food!
Yu Garden/Huxinting Teahouse.
佳家 for 小龙包.
小样 (Little Yang's) for 生煎包 (sheng jian bao). Fried soup filled dumplings. Think skin crunchy bottom texture.

Links - August 11, 2015

Avy Faingezicht — Tue, 11 Aug 2015 03:00:00 GMT

How to Use Heroku PGBackups

Brightball Articles — Tue, 11 Aug 2015 03:00:00 GMT

Backing up your data is one of the most critical activities for your application. Heroku PGBackups makes the entire experience pretty simple but comes with a lot of flexibility too, with a number of options for smooth restoration.

A Culture of Quality

Ben E. C. Boyter — Mon, 10 Aug 2015 11:17:33 GMT

The best working environment I had the pleasure to work in had a strong emphasis on testing and software quality in general. Product teams were encouraged to spend extra time ensuring that everything worked over shipping before it was ready. The transformation it went through was incredible. Having come from a culture very much wild west through to where it was. An example of the advantages this brought was that before adopting this culture website launches were a traumatic event. Teams would block out 48 hours stretches and work solid fixing bugs at go live. Very stressful for all involved and not a healthy working environment. Contrast to where things ended up where several websites were launched on the same afternoon without a hitch. Very impressive considering the scale of the websites being dealt with (several million uniques a day).

Singapore job board bookmarks

xenodium.com @alvaro — Mon, 10 Aug 2015 03:00:00 GMT

$100 startup

june.kim — Mon, 10 Aug 2015 03:00:00 GMT

Adventures after Google I/O-4(Databinding)

Prasham H Trivedi — Mon, 10 Aug 2015 03:00:00 GMT

After trying and playing some new stuffs(except M preview) released in Google I/O, I have found some trivial things to take care of. In concluding part of series we will see what are some issues with Google’s Databinding plugin and how we can get around it

RAM is the new disk – and how to measure its performance – Part 1 – Introduction

Tanel Poder Blog — Mon, 10 Aug 2015 02:26:55 GMT

[ part 1 | part 2 | part 3 ]

RAM is the new disk, at least in the In-Memory computing world.

No, I am not talking about Flash here, but Random Access Memory – RAM as in SDRAM. I’m by far not the first one to say it. Jim Gray wrote this in 2006: “Tape is dead, disk is tape, flash is disk, RAM locality is king” (presentation).

Also, I’m not going to talk about how RAM is faster than disk (everybody knows that), but in fact how RAM is the slow component of an in-memory processing engine.

I will use Oracle’s In-Memory column store and the hardware performance counters in modern CPUs for drilling down into the low-level hardware performance metrics about CPU efficiency and memory access.

But let’s first get started by looking a few years into past into the old-school disk IO and index based SQL performance bottlenecks :)

Outside Lands

Avy Faingezicht — Sun, 09 Aug 2015 03:00:00 GMT

My second visit to the music festival. It did not disappoint.

How to sign release apk with gradle

zserge's blog — Sun, 09 Aug 2015 03:00:00 GMT

Whenever I prepare my APK for the release - I used to do the following steps: Copy keystore to the project directory Copy gradle.properties into the project directory with keystore name, alias and password defined Copy some code from some blog that would load the properties and set up the signing Finally, do gradle build But why not to move the boilderplate out of build.gradle? Helper signing script I wrote a tiny helper script that can be included into the projects:

Update Raspberry pi firmware from Kali Linux

Shyam Jos — Sun, 09 Aug 2015 03:00:00 GMT

We can easily update Raspberry pi firmware in raspbian by running the command rpi-update, But in kali linux we dont have rpi-update command by default.

To install the rpi-update in kali, run the following command:

sudo curl -L --output /usr/bin/rpi-update https://raw.githubusercontent.com/Hexxeh/rpi-update/master/rpi-update && sudo chmod +x /usr/bin/rpi-update

Then, to update your rpi firmware, run the following command:

sudo rpi-update

Now restart your rpi to new firmware to take effect.
For more Expert options visit the rpi-update project page

Germany travel bookmarks

xenodium.com @alvaro — Sat, 08 Aug 2015 03:00:00 GMT

Menorca travel bookmarks

xenodium.com @alvaro — Sat, 08 Aug 2015 03:00:00 GMT

Travel tools bookmarks

xenodium.com @alvaro — Sat, 08 Aug 2015 03:00:00 GMT

Philippines travel bookmarks

xenodium.com @alvaro — Sat, 08 Aug 2015 03:00:00 GMT

Wooden Rack

june.kim — Sat, 08 Aug 2015 03:00:00 GMT

2015-08-07

ho.dges.online — Fri, 07 Aug 2015 03:00:00 GMT

The Chosen One

Kaizen

june.kim — Thu, 06 Aug 2015 03:00:00 GMT

How Palette was made

zserge's blog — Thu, 06 Aug 2015 03:00:00 GMT

Palette is an open source lightweight color picker for the most famous color schemes, like Material Design, Flat UI, iOS, Windows Phone etc. Despite being color blind, I like apps and websites with the right selection of colors. And sometimes when starting a new project it’s easier to pick a color palette from the existing ones. Or, if you want to meet the guidelines for the certain platform - you’d better use the colors they offer.

C# as a Language from old Google+ Post

Ben E. C. Boyter — Thu, 06 Aug 2015 01:37:38 GMT

The more I use C# as a language for writing things the more I am convinced that its approach really is the best language approach out there.

The unit test support is excellent which allows development speed to be just as fast as any dynamic language (Python, PHP, Perl).

The static typing catches so many issues before you get to runtime and allows sweeping changes without breaking things.

Unlike Java it has the var keyword (saves time and improves readability) and so many more useful functions which yes you can replicate but are just built in and work correctly.

Recombination

june.kim — Wed, 05 Aug 2015 03:00:00 GMT

Slack bot for Phabricator Notifications

Srijan Choudhary, all posts — Tue, 04 Aug 2015 21:20:00 GMT

Setting up a slack bot for phabricator

2015-08-04

ho.dges.online — Tue, 04 Aug 2015 03:00:00 GMT

Both boys very tired at the end of a long evening and wanting to sleep.

Pee-training Rigel

Liza Shulyayeva — Mon, 03 Aug 2015 19:58:47 GMT

Note: This is part of a series of posts about Rigel the Maine Coon kitten who was given a urinary infection by a veterinarian in Stockholm during a routine surgery and has been struggling with peeing since. You can read more in the Rigel section.

The Congress

🦉 olshansky 🦁 — Mon, 03 Aug 2015 16:30:41 GMT

Olshansky's review of The Congress

Problem Solving 2

june.kim — Mon, 03 Aug 2015 03:00:00 GMT

Upgrading Linode II: Xen to KVM

Thomas Hunter II — Mon, 03 Aug 2015 03:00:00 GMT

Making army lists in Age of Sigmar

Ben Overmyer's Site — Mon, 03 Aug 2015 03:00:00 GMT

Making an army list in Age of Sigmar is an unusual process for those of us used to the traditional Games Workshop methods. In Warhammer Fantasy you decided on a points value, an army, and then tried to match a unit selection based on force organization rules and a balanced strategy to defeat as many types of opponents as possible. Those opponents would most often be similarly organized. In Age of Sigmar, however, there are only two ways to build for “strength:” by sheer force of numbers, or by maximizing keyword synergies. The sheer-force-of-numbers (a.k.a. pay-to-win) approach will not win you any friends, and may get you banned from games. The keyword synergy approach, though, is as yet an undiscovered country. Everyone is starting to pick up on the idea that you can take certain units to play on their collective strengths. As the community finds its feet in this new game, strong strategies are bound to come out.

For example, the Dark Elf Cold One Knights are somewhat lackluster by themselves. However, in the presence of a Dreadlord on Cold One they (and Cold One Chariots) reroll charge rolls and add 1 to Wound rolls for one of their attacks. It's combinations like this that will define Age of Sigmar army lists. While you can make all kinds of interesting setups, it's this system that's going to draw the tactical minds out of its players. I'm reminded heavily of Magic: the Gathering or Hearthstone here. Both of those games encourage players to build winning strategies on specific synergistic themes that amplify their benefits, while not discouraging taking completely odd selections for flavor or for backup strategies. For my part, I intend on running a lot of Dark Elves (or Exiles, as they're now called) as the core of my first army. There's a lot to love there.

Offering free internet well almost and survive disaster, power outage, Internet blackout

Fabrice Aneche — Sun, 02 Aug 2015 23:26:38 GMT

In my neighborhood, I’m experimenting to give free access to some services, not the full internet but a full access to Wikipedia, maps… Here are some tips do to the same. Getting access to the internet may be crucial for our lives, but the commercial providers are here to make money out of it. They don’t event provide disasters nor minimum safety access. Sometimes even in normal condition, while traveling your roaming will cost you hundreds just to get access to a map or Wikipedia.

Access OS metrics from Golang

Fabrice Aneche — Sun, 02 Aug 2015 20:10:38 GMT

I’ve recently published StatGo, it gives you access to your operating system metrics like free memory, used disk spaces … It’s a binding to the C library libstatgrab, a proven stable piece of code that works on many different systems, FreeBSD, Linux, OSX … It’s very simple to use: s := NewStat() c := s.CPUStats() fmt.Prinln(c.Idle) 98.2 Feel free to contribute, it may need some improvement but it’s working I’m using it in a small metrics web server to monitor small network of servers.

Laravel log file backups to S3

Liza Shulyayeva — Sun, 02 Aug 2015 15:51:05 GMT

SnailLife does a lot of logging for debugging purposes. Aside from the general laravel.log I have separate loggers for each snail and also write logs for stuff like deleted items etc in separate files.

Host your blog on Github with autodeploy

Fabrice Aneche — Sun, 02 Aug 2015 07:32:43 GMT

I’ve always developed my own blog system, that’s a good way to learn a new langage. But having to maintain a working server or hosting is no fun, there are some solutions like Jekyll or Hugo they generate static web pages based on some Markdown files you wrote. As it’s just basic html files, they can be served by Github gh-pages. It opens the door to blogging from anywhere without internet connection or your own laptop, just write some Markdown then publish to github later or event edit your new blog post from the Github editor.

Migrate to Hugo

Fabrice Aneche — Sun, 02 Aug 2015 05:36:51 GMT

This blog is running Hugo with an auto deploy via Wercker and hosted on Github Page. Take #2 Previous blog was hosted on Google App Engine with a Python blog system, to get the previous articles, I had to run in a small data migration. Make a backup from GAE admin web interface: Go to Datastore admin and backup your entity mine was Post and then backup to blobstore, go to Blob Viewer and download your file named around datastore_backup_datastore_backup_2015_08_02_Post-157413521680733022360302ADC43E4-output-1-attempt-1

Problem Solving

june.kim — Sun, 02 Aug 2015 03:00:00 GMT

Unit Tests Manage Complexity

The Cranky Developer on Crater Moon Development — Sun, 02 Aug 2015 03:00:00 GMT

Unit testing is more than a design tool, not just a practice that enhances quality, it is a tool for managing complexity.

His love endures forever!

trv.rs — Sun, 02 Aug 2015 03:00:00 GMT

Give thanks to the God of heaven, for his steadfast love endures forever. (Psalm 136:26 ESV)

My Father, I have always loved this anthem of a Psalm, but today through it I praise you still deeper. Thank you for what you showed me buried in its treasure-trove this morning. I love the list of great works, and have always triumphantly recounted with the psalmist your wonders of old: victory over enemies, freedom from unjust oppression, power over creation's bounds and the rules of physics itself… but for some reason I have missed the whole point of the song. These actions describe You yes, but the psalmist calls us to joyous praise because of a still deeper truth: Your love endures forever! He recounts your works in Israel's history with joy, but to him they are a side note to the work of redemption. Thank you that you love endures, it was conceived, begun, planned in the time before time, it continued, was promised, executed and brought to life in the past, and continues gloriously on into eternity, for Your love has a name and His name is Jesus. Thank you that your love endures… for without it I certainly will not. God you are great. You are truly awesome. Worthy is your name! Hallelujah!

Simple Python to Batch search CAZy Database

Erico Notes — Sun, 02 Aug 2015 02:41:21 GMT

This code may be old. please follow here to the most up-to-date version: https://github.com/ericoporto/CazyBatchSearch

Yes, a way to use a table to search the Carbohydrate-Active enZYmes Database. Below the Python 2.7.3 script.


'''
Copyright 2014 Érico Porto

Licensed under the Apache License, Version 2.0 (the "License");
you may not use this file except in compliance with the License.
You may obtain a copy of the License at

    http://www.apache.org/licenses/LICENSE-2.0

Unless required by applicable law or agreed to in writing, software
distributed under the License is distributed on an "AS IS" BASIS,
WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
See the License for the specific language governing permissions and
limitations under the License.
'''
from __future__ import print_function  
 import httplib  
 def findEnzymeInCazy(enzyname):  
   conn = httplib.HTTPConnection("www.cazy.org", 80)  
   conn.connect()  
   conn.request('GET', "/search?tag=4&recherche=" + enzyname)  
   lines = conn.getresponse().read().split('\n')  
   for i,line in enumerate(lines):  
     if ( '<tr><td><a href="http://www.cazy.org/' in line ):  
       linkLine = lines[i]  
       j = linkLine.find( '"http://www.cazy.org/' ) + 21  
       k = linkLine.find( '.html"', j )  
       return linkLine[j:k] # beteween first and second double quotes  
   return None  
 f = open('saida.csv','w')  
 print("enzima     ; rankdomal", file=f)  
 print("enzima     ; rankdomal")   
 for line in open("enzylist.csv","r"):  
   pieces = line.split(";")  
   enzyname = pieces[0].strip()  
   enzyInCazy = findEnzymeInCazy(enzyname)  
   if ( enzyInCazy != None ):  
     print(enzyname.ljust(15) + '; ' + enzyInCazy.ljust(20), file=f)  
     print(enzyname.ljust(15) + '; ' + enzyInCazy.ljust(20))  
   else:  
     print(enzyname.ljust(15) + '; ' + "not found!", file=f)  
     print(enzyname.ljust(15) + '; ' + "not found!")

So, you will need a file named enzylist.csv for this to work. Here take this:

 BAI68730
 BAH05588
 ACV62532
 EAL90874
 ABG47447
 ACT04224

So if you run it, you should get something like:

 python .\enzymtable_simple.py  
 enzima     ; rankdomal  
 BAI68730    ; GH57  
 BAH05588    ; GH94  
 ACV62532    ; GH94  
 EAL90874    ; GH13  
 ABG47447    ; GH18  
 ACT04224    ; GH51

So all should be good. Yes I run Python on Windows 8 using PowerShell. Just add the Python folder to the PATH variable, and all should run well - and also, PowerShell is much faster and easy to use than CMD.

More history, some musing about syncing files

Negative Feedback — Sat, 01 Aug 2015 10:01:50 GMT

It's interesting how your tools affect your choices and how you can end up in a place which works for your current needs but doesn't make the next step you're trying to get to easy.

To understand my current situation regarding managing my files, it's probably best to go into a little bit of history.

Early computer use: no syncing

I enrolled as an undergraduate in 1997. When I arrived on campus I was given my very first e-mail address and I remember spending huge amounts of time in the computer labs writing to all of my friends (using a program called Pine, which looked like this):

Pine screenshot

Files were small and you carried them around with you on things that we called stiffies. The Americans called them 3½-inch floppy disks. The capacity of a stiffy was 1.44 MB. This was enough for most of your work in a semester. Stiffies were notoriously unreliable, so you never really worked on the disk only, because for all you knew, they could die tomorrow. Stiffies were for moving files from one computer to another, not for long-term storage. Some people had CD writers, but campus computers didn't.

I didn't have a computer in my res room until my second year, and when I did it wasn't connected to any kind of network until third year. I could only check my mail and do IRC on campus computers.

Once I got a computer in my room, an issue emerged: sometimes I would start working on something in my room and I would need to continue working on it on campus. So I would copy it to a stiffy and ride my bike to campus, where I would copy it over and keep working. This process led to copies of the files on both machines with no clear way to keep track of which one I had worked on last. So if I forgot my disk on my desk and worked on the campus copy instead of the version I had in my room, things would break. Luckily files were small at first, but technology rapidly improved and by final year (2000) we had CD writers. At this point, we had Internet connections, but they were so slow that it was much faster to burn a CD and bike to campus than to send the data over the Internet. The fastest dial-up modem speed was 56 kbps, which would require more than a day to transfer the 700 MB that fits on a CD.

For my entire undergraduate career, therefore, I could never sync my account on campus with my computer at res or the one at home. I just had some random files in various places which I had to transfer piecewise.

Postgraduate: two computers, external HDD

When I enrolled for my Masters in 2001, I moved into a lab and got assigned a computer. Now I had a computer in my room at res and one at the lab and I often found myself having to transfer files between them. I installed a removable hard-drive bay enclosure and had a hard drive that I would shuttle between the two locations. Moving things this way was a big deal. IDE hard drives don't support hot-swapping, so I would sync to the hard drive, turn off my computer, remove the hard drive and then boot up with the hard drive in place at the new location. I believe that this hard drive was either a 500 MB or 1 GB. It is around this time that I started using Unison to sync the external hard drive to the machine where I was working. This meant I could survive forgetting to bring the hard drive to the lab, since Unison stores a hash of all the files and can recognise if two files have changed. I synced basically all of my files except for music files and movies, which were too large to fit on this external hard drive.

Two desktops, laptop, local network

When I started working for the University of Pretoria in 2003, I continued to use the HHD sync strategy for a while, and when I got a laptop, I started treating it similarly to the HHD - I would use Unison, but now using the (wired) network to connect the machines instead of plugging in a hard drive. I continue to use this strategy today. Every morning before starting to work on my desktop, I sync. Every evening before leaving, I sync. This way I have a complete copy of my working files on both machines (my laptop and my desktop). This provides a level of redundancy (I don't think of this as a backup strategy, that's handled by Bacula on my desktop and these days by Time Capsule on my laptop).

A change: cheap Internet

During my first couple of years at the university, we were charged R1/MB for internet access. Let me repeat that in units that make more sense to a modern reader: that's R1000/GB. This means that any kind of large file transfer using the Internet was completely out of the question. It also led to me turning into a packrat. Since downloading software or even articles was so expensive, I developed a huge folder of downloads that I would maintain to avoid having to re-download files.

Rummaging around my old e-mail, I find that in 2005 we had a limit of R60/month in place. That's right, ten years ago the University limited my monthly bandwidth to 60 MB. In 2011 we were being charged 16 c/MB (R160/GB) on student access, but staff caps had been dropped.

Dropbox launched at the end of 2008, registering dropbox.com in 2009. This is when the idea of easily syncing files over the Internet really started taking hold. I think I installed Dropbox in 2010, when our IT department dropped caps.

At the moment I use Dropbox for sharing files with other people and I still use Unison to handle syncing the files on my laptop and my desktop.

A remaining issue: campus proxy/firewall

Even though we are no longer charged for Internet access on campus, we still have to log in to our proxy server to get access to the Internet. This does not work transparently for all applications. In addition, for obvious reasons, we can't set up a server on a machine inside campus which will be visible off-campus. I have only recently (early 2015) been able to set up a virtual machine inside our DMZ which would allow me set up a central repository for files.

I have to switch Dropbox between proxy and no proxy every time I move from home to campus.

Side-effects

Because I built my syncing strategy when Internet access was either slow or expensive or both, and because I've been working on two different computers regularly, I have avoided dependence on client-server technologies. Specifically, I have not depended on a client-server database like MySQL or PostgresSQL. Instead, when I required SQL database functionality, I've use SQLite, which allows me to continue with the file-based syncing strategy.

But now I've started investigating NoSQL technologies like Mongodb or Redis (and later, perhaps something like Hadoop). It doesn't appear as though these things have a simple file-based alternative similar to SQLite.

Why change what works?

It is clear from my interaction with industry that their model is very solidly built around central databases with access from clients. This is a similar model to all the Internet services we use now. I'm familiar with setting up these systems, but my students and I often have only intermittent access to the the Internet. The students because they can't afford it, myself because I travel using the Gautrain, and often have to work in a disconnected state. Not to mention load shedding.

Now I am interested in building a better system which has some of the nice properties I have come to rely on with Unison, or at least allows for limited offline use with periodic syncing. I know this will not be as simple as setting up two directories to sync, but I'm getting the horrible feeling that each system will have its own problematic story and that the central server strategy is just a way of avoiding sync rather than a solution.

If anybody has a suggestion, I'd be pleased to hear it.

Justification After Actions

Venam's Blog — Patrick Louis (Lebanon) — Sat, 01 Aug 2015 00:00:00 GMT

Hello fellow readers, In this post I'll discuss a subject that has been nagging me. Justification After Actions I usually write to clear my mind about a topic. These days I can't really keep up with all the things that bug me. This subject is a preparation to a broader subject that I'm preparing to write about.

Automated mugshots with Python and OpenCV

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Fri, 31 Jul 2015 20:15:15 GMT

One of my most favourite books, Cryptonomicon by Neal Stephenson contains a scene in which one of the protagonists, Randy, is letting other people use a demo on his laptop in a business meeting. Since Randy does not know all the people in the meeting he figures it would be a good idea to clandestinely collect their mugshots by writing a program that captures all the different faces appearing in front of his webcam.

I always wanted to do this because it sounds like a very fun idea. I sometimes leave my laptop open and lock it when I am on a conference and know that I shall be shortly returning to my seat. It was always a pet peeve of mine to know whether other people were trying to unlock the laptop…

So, with the power of Python and OpenCV, years after reading Cryptonomicon for the first time, I finally wrote my own version of Randy’s program. This turned out to be surprisingly easy. It is a testament to the versatility of Python (and its libraries of course) that this problem can be solved by a really short program.

You can download the script from its GitHub repository. Further improvements may include (in the hopes that I get around to do this):

Recognizing and categorizing different faces
Taking mugshots when certain actions happen (a git hook could trigger this, for example)
Investigating different methods of face recognition
Detecting basic emotions

The script is dead simple to use. Just call

./mugshot.py

./mugshot.py --hide

in case you do not want to get visual feedback about the recognized faces. mugshot.py uses a simple logic to not fill about its working directory with useless faces: If either the number of faces changes between two frames or more than 10 seconds passed between the current face detection and the previous one, a new mugshot will be taken. The mugshots are named using the current time, with a suffix in case multiple faces are detected.

Have fun!

Dataflow Engine

Matt Keeter — Fri, 31 Jul 2015 03:00:00 GMT

Designing with graphs

JIT Development

june.kim — Thu, 30 Jul 2015 03:00:00 GMT

Making Stuff

june.kim — Thu, 30 Jul 2015 03:00:00 GMT

Implementing AES

Krishna's blog — Thu, 30 Jul 2015 03:00:00 GMT

This article explains how to implement the Advanced Encryption Standard (AES) algorithm

Break hard, break early.

june.kim — Wed, 29 Jul 2015 03:00:00 GMT

"Facebook Background Task Host" consuming 100% disk bandwidth

Databases & Life — Wed, 29 Jul 2015 03:00:00 GMT

The program “Facebook Background Task Host” was consuming the entirety of my disk bandwidth on my Windows 8.1 computer. Further, there seemed to be almost nothing written about this program on the internet according to Google. Rebooting the computer didn’t help: after the reboot it was there again. I did not remember downloading anything from Facebook to run on my computer.

The solution was to go to the Windows 8 start screen, and find the Facebook “app”, and then uninstall it.

keybase.io

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Tue, 28 Jul 2015 22:07:33 GMT

I now also have keybase.io profile and feel slightly overwhelmed by the possibilities. So far, I managed to verify my website, my public key, and my GitHub identity. This way of keeping tabs about a person using a single identifier has certainly its merits, but supposing I were to use, say, Hacker News, it might not be in my interest to have that profile associated with my public persona.

Further reports about my experiences are almost certain to follow.

The Missile Man

Honest Musings — Tue, 28 Jul 2015 06:30:14 GMT

An eagle soared in the blue sky, Unhindered and mightily he rose and fell, With the soiled humanity below his feet, Looked down upon and called him King. A glint of metal broke his reverie, Flustered he called, Who goes there?, Don’t you know this kingdom belongs to me, Speak and say, Who sent thee? My father, the … Continue reading

What's your dream job?

june.kim — Tue, 28 Jul 2015 03:00:00 GMT

Golang parallelism issues causing “too many open files” error

The Technical Blog of James on purpleidea.com — Mon, 27 Jul 2015 14:39:45 GMT

I’ve been hacking in golang for a while, but I’ll admit that I didn’t get too deep into some of the language nuances until more recently. Since some of them have started to bite me, here’s a little post-mortem of one of the problems I was having.

After hacking and testing code all day, I made a seemingly innocuous change, and when running my program, I saw the following error:

2015/07/10 14:34:12 too many open files

I didn’t know what I broke, but it was obviously my fault. I reverted my recent changes, but still the error persisted. Internet searches and many painful hours of debugging ensued.

A/B Testing

Ben E. C. Boyter — Mon, 27 Jul 2015 11:11:59 GMT

A/B testing is testing the comparison of two outputs where a single unit has changed. It is commonly used when when trying to increase conversion rates for online websites. Also known as split testing an example would be trying to increase user clicks on a specific button on a website. You may have a theory that red buttons work better then green. You would try out both against real users and see which one performs better.

Firewalld blocking snmp and fixing it (with Ansible)

zufallsheld — Mon, 27 Jul 2015 11:00:00 GMT

Some time ago I had to use a new CentOS 7 virtual-machine for some things that aren’t relevant to this post. This machine had to run all the time but also had some problems with a sporadically failing application server. That’s why I decided to monitor the machine …

Al Jazeera Interview with Matt Crampton

MattCrampton.com — Mon, 27 Jul 2015 08:00:00 GMT

Al Jazeera Interview On July 27th, 2015 I spoke with Al Jazeera about the Gig economy and trends for millenials and companies.

Past, present and future

june.kim — Mon, 27 Jul 2015 03:00:00 GMT

angular-fng library - Improving the performance of large apps

Writing on Adam Craven — Mon, 27 Jul 2015 03:00:00 GMT

How to avoid root scope digests using faster angular events, which mimic the functionality of the existing ng-event directives, but have a feature that allows them to be called in a desired scope, rather than trigger a root scope digest.

A guide to analyst relations for startups

CRAIG KERSTIENS — Sat, 25 Jul 2015 23:55:56 GMT

When it comes to go to market and marketing there’s lots of pieces in a toolchest that all work together. One that comes a bit later, but if used properly (much like a PR agency) can be valuable is industry analysts. And while working with a PR agency can quickly start to become clear. How to work with analysts so it is productive on both sides can take a bit longer to figure out, or at least it did for me. Even before you do start working with them there’s the question of if or when should you. Here’s hoping this primer makes it a bit faster and easier for others.

What is an analyst

Apologies to all analysts, but of all parts of this post I might butcher this one

Analysts talk to a lot of companies, both the ones making products as well as the ones purchasing them. I’m not actually sure what the spread is I’d guess 80-20. A large output of this and other activities is creating various reports and rankings. Gartner’s Magic Quadrant is probably the most well known industry ranking. Much of what they create isn’t freely available for consumption so you likely don’t see the sheer volume of insights they put out.

Why would you engage with an analyst

So what do they do for you? There’s really two major buckets:

Help with sales/marketing - Given they’re informing and influencing buying decisions of businesses they can be one more person on your side. If a launch in Techcrunch makes business foo aware of product bar, then an analyst report or ranking can help sway a decision on whether to try bar vs. baz.

Consulting - The other major opportunity is for the analyst to give some form of guidance. In a larger company when you already have an established product they should absolutely be part of your launch process (more on that in a future post). They’re actively following your market and space, hopefully just as you are to some extent. They can offer an outside perspective and help with broad areas of focus and messaging.

More - In reality it’s as clean cut as above. They may be able to introduce you to good candidates for hiring. They may be able to introduce you to a large company interested in acquiring some capability which you have. They may be able to connect you with investors. All of these things can and do happen, but the above buckets typically are the primary drivers.

When?

First, engaging with analysts should always come after you have some confidence in the product, after you’ve started some marketing drumbeat, and after sales. In short don’t be in a huge rush here, you’ll get there, but don’t be in too big of a rush. As you start to get some attention and momentum it’s just as likely they’ll engage with you first as you reaching out. Also, marketing != sales, more on that in a future post.

But, let’s assume you’ve got a product which targets business (Analysts aren’t just for tech companies, though you’ll see the benefit here sooner if you’re say a database company than a HR product). Let’s also assume you’ve got some sales and have some good launches under your belt. As it starts to come up in sales calls if you’re in any industry reports or rankings that may be an indicator, if you’re hearing about other competitors having more validation in such reports. As a general rule of thumb once you’ve got inhouse PR they should be able to help guide and steer to the right time.

So how do you engage with an analyst

If you’re engaging in some form of report or article, that should start to be pretty self explanatory. They’ll send you a questionnaire, you fill it out. You go back and forth a little bit.

However, the majority of my interactions aren’t on those articles and reports, for ever one time I fill out lots of questions to help some report or ranking I have 20 calls with an analyst.

There are two primary calls you can have, an inquiry and a briefing.

Inquiries

Inquiry is just a fancy word for consulting call. An inquiry you will always be paying for.

A small detour here. The regularity and consistency in which you engage with an analyst makes a difference. They’re also people at the end of the day, so while firms have certain styles it’s even further multiplied by being very people driven. In your interactions you’ll have a different rapport with different people, it’s at a minimum important to be aware of this.

So back to an inquiry. Within an inquiry your goal is to pull back the curtain and give some backstage insights into what you’re doing and where you’re headed. This is typically under NDA and trust the NDA of an analyst. It’s worthwhile to be as candid as you can here, yes it feels weird, but you’ll get the most value. They’re not like that of a reporter looking for a scoop (not that you can’t trust reporters, just know if you say it, it’s on record). You don’t have to relish the entire call to one area, but areas of coverage are often:

Upcoming products and major releases you’re working on
Broader strategy and roadmap
Get input on what they’re seeing and hearing from customers

Briefings

The other type of call we have is a briefing. This is a little similar to that of a press briefing. You’ll get on the call, and walk through some upcoming launch or just give an update on your company and progress. The latter is more common if they’re unfamiliar with you or your product.

Analyst briefings are good to do earlier than your press briefings, compared to press they’re like a bike with training wheels. It’s best if you still maintain your balance–the ride will be smoother, but there’s a little less risk of completely toppling over. One key difference is you often have a powerpoint deck you get to walk through during an analyst briefing. I’ve found this is helpful for pacing and key messages, I used to be skeptical, but now very much feel it’s always worth doing.

Pro-tip: You can create a deck and use it for press too, no they won’t want to get on a gotomeeting, but you can send it over so they have more content later. BUT, more importantly you can also walk through it on your own screen if it helps with pacing.

Within a briefing you’ll have some ability to ask them questions at points. Does this resonate? Are you hearing similar? What are you seeing in the market? Don’t turn it into an inquiry, but knowing the parts that hit home for them allow you to refine your pitch for the next call.

Engaging - the tactical parts

“Analysts are pretty much paid to talk and write” - @cote. So expect that often when you occupy their time there’s a price to it. In terms of finding them it should be pretty easy, to know the list of ones in your space, you may see them quoted or reference in various media outlets. You may just naturally crop up in a report.

If you create a regular relationship with them you’ll have some contract of hours over the course of a quarter or year. At an early stage company this is often owned and manage by whomever runs your PR from an internal perspective.

Conclusion

If you’re about to engage with analysts for the first time or haven’t figured out how to get the most out of your interactions I hope the broad overview is helpful. If there’s some glaring parts you feel I’ve missed let me know @craigkerstiens. And for further reading/watching I’d encourage checking out the great talk from @cote in the Heavybit library.

As far as take-aways and a recap:

Don’t be too eager to jump in with analysts. They can absolutely provide value, but you have to put some time in before it really starts to pay off. It’s not an overnight change and takes building a rapport with them.
At the same time, analysts can be useful in many B2B areas not just tech ones.
When in an inquiry be open and as transparent as possible.
Powerpoint/Keynote/Google presentations are useful in briefings, even if it’s just for you to follow along.

SnailLife messaging system

Liza Shulyayeva — Sat, 25 Jul 2015 23:43:17 GMT

“What’s SnailLife”, you say? Well I’m glad you asked! My snail simulation has gone through a couple of names…and even though I was never 100% happy with Gastropoda it was the best I could come up with - a name that was unique and didn’t allow the project to sound too “gamey” (because it’s not a game). All of the cooler names I could come up with weren’t suitable for various reasons (like domain name availability and such).

Time Lapse

🦉 olshansky 🦁 — Sat, 25 Jul 2015 16:26:44 GMT

Olshansky's review of Time Lapse

Piece by Piece

Josh Thompson — Sat, 25 Jul 2015 03:00:00 GMT

The following is inspired by Amy Hoy.

I’ve got a secret to share: I’m working on building a product (of the digital variety) that will be so damn goodpeople will pay me $100 or more to get it.

I’ve got a lot of bits and pieces of it littered around the internet, my computer, and my head. If you want a sneak peak at an earlier (unsuccessful) stab at this, click over to www.belaybetter.com.

There you can buy a PDF for about $20, if you wanted to, to read about 25 pages of my thoughts and ideas on dealing with fear while lead climbing. I’ve sold a few copies (maybe eight?) over the last eighteen months .So, the ROI on the time invested in that particular product is hovering not much over $1/hr.

The path to get from here to there is daunting:

What is “there”? I’ve got it loosely defined, but need to tighten that up)
I need a way better product.
Do I need to add “services” to the product?
If so, what are those? Videos, an hour or two of my time via Skype
How do I know when I’ve reached a product that is good enough
Assuming I’ve got something good enough, I still need to build a much better landing page.
And I probably need an email drip campaign to build trust with subscribers.
Subscribers. Where do I get them? Reddit? Guest posts?
Guest posts. I need to write to the owners of well-trafficked and run some of the stuff I’ve made by them to see what they thing/ways I can make it better.
I need a list of potential authors to reach out to.

But guess what? Every one of those steps is doable. By me. I just need to sit down and start chewing through them. Some of it involves thinking, some of it involves writing, and some of it involves emailing. There’s nothing more complicated than that.

Private Social Network

Mauricio Uribe | m.x.uribe — Fri, 24 Jul 2015 07:00:00 GMT

Some benefits of a private social network

iSCSI Benchmarking

smcleod.net — Fri, 24 Jul 2015 03:00:00 GMT

The following are benchmarks from our testings of our iSCSI SSD storage.

67,300 read IOP/s on a VM on iSCSI

(Disk -> LVM -> MDADM -> DRBD -> iSCSI target -> Network -> XenServer iSCSI Client -> VM)
Per VM and scales to 1,000,000 IOP/s total

root@dev-samm:/mnt/pmt1 128 # fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=128 --size=2G --readwrite=read
test: (g=0): rw=read, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
2.0.8
Starting 1 process
bs: 1 (f=1): [R] [55.6% done] [262.1M/0K /s] [67.3K/0  iops] [eta 00m:04s]

38,500 random 4k write IOP/s on a VM on iSCSI

(Disk -> LVM -> MDADM -> DRBD -> iSCSI target -> Network -> XenServer iSCSI Client -> VM)
Per VM and scales to 700,000 IOP/s total

root@dev-samm:/mnt/pmt1 # fio --randrepeat=1 --ioengine=libaio --direct=1 --gtod_reduce=1 --name=test --filename=test --bs=4k --iodepth=128 --size=2G --readwrite=randwrite
test: (g=0): rw=randwrite, bs=4K-4K/4K-4K, ioengine=libaio, iodepth=128
2.0.8
Starting 1 process
bs: 1 (f=1): [w] [26.3% done] [0K/150.2M /s] [0 /38.5K iops] [eta 00m:14s]

Raw device latency on storage units

Intel DC3600 1.2T PCIe NVMe

root@s1-san6:/proc  # ioping /dev/nvme0n1p1
4.0 KiB from /dev/nvme0n1p1 (device 1.1 TiB): request=1 time=104 us
4.0 KiB from /dev/nvme0n1p1 (device 1.1 TiB): request=2 time=83 us
4.0 KiB from /dev/nvme0n1p1 (device 1.1 TiB): request=3 time=51 us
4.0 KiB from /dev/nvme0n1p1 (device 1.1 TiB): request=4 time=71 us

SanDisk SDSSDXPS960G SATA

root@pm-san5:/proc  # ioping /dev/sdc
4.0 KiB from /dev/sdc (device 894.3 GiB): request=1 time=4.2 ms
4.0 KiB from /dev/sdc (device 894.3 GiB): request=2 time=4.1 ms
4.0 KiB from /dev/sdc (device 894.3 GiB): request=3 time=4.1 ms
4.0 KiB from /dev/sdc (device 894.3 GiB): request=4 time=4.1 ms

Micron_M600_MTFDDAK1T0MBF SATA

root@pm-san5:/proc  # ioping /dev/sdf
4.0 KiB from /dev/sdf (device 953.9 GiB): request=1 time=157 us
4.0 KiB from /dev/sdf (device 953.9 GiB): request=2 time=190 us
4.0 KiB from /dev/sdf (device 953.9 GiB): request=3 time=65 us
4.0 KiB from /dev/sdf (device 953.9 GiB): request=4 time=181 us
```shell
## Latency on the a VM

- (Disk -> LVM -> MDADM -> DRBD -> iSCSI target -> Network -> XenServer iSCSI Client -> VM)

```shell
root@dev-samm:/mnt 127 # ioping pmt1/
4096 bytes from pmt1/ (ext4 /dev/xvdb1): request=1 time=0.6 ms
4096 bytes from pmt1/ (ext4 /dev/xvdb1): request=2 time=0.7 ms
4096 bytes from pmt1/ (ext4 /dev/xvdb1): request=3 time=0.7 ms

--- pmt1/ (ext4 /dev/xvdb1) ioping statistics ---
3 requests completed in 2159.1 ms, 1508 iops, 5.9 mb/s
min/avg/max/mdev = 0.6/0.7/0.7/0.1 ms
root@dev-samm:/mnt  # ioping pmt2/
4096 bytes from pmt2/ (ext4 /dev/xvdc1): request=1 time=0.6 ms
4096 bytes from pmt2/ (ext4 /dev/xvdc1): request=2 time=0.8 ms

--- pmt2/ (ext4 /dev/xvdc1) ioping statistics ---
2 requests completed in 1658.4 ms, 1470 iops, 5.7 mb/s
min/avg/max/mdev = 0.6/0.7/0.8/0.1 ms
root@dev-samm:/mnt  # ioping pmt3/
4096 bytes from pmt3/ (ext4 /dev/xvde1): request=1 time=0.6 ms
4096 bytes from pmt3/ (ext4 /dev/xvde1): request=2 time=0.9 ms
4096 bytes from pmt3/ (ext4 /dev/xvde1): request=3 time=0.9 ms

Git archive with submodules and tar magic

The Technical Blog of James on purpleidea.com — Thu, 23 Jul 2015 23:32:27 GMT

Git submodules are actually a very beautiful thing. You might prefer the word powerful or elegant, but that’s not the point. The downside is that they are sometimes misused, so as always, use with care. I’ve used them in projects like puppet-gluster, oh-my-vagrant, and others. If you’re not familiar with them, do a bit of reading and come back later, I’ll wait.

I recently did some work packaging Oh-My-Vagrant as RPM’s. My primary goal was to make sure the entire process was automatic, as I have no patience for manually building RPM’s. Any good packager knows that the pre-requisite for building a SRPM is a source tarball, and I wanted to build those automatically too.

Add site-specific browsers to your workflow

xenodium.com @alvaro — Thu, 23 Jul 2015 03:00:00 GMT

There are three browser tabs continously used in my workflow: GMail, Google Calendar, and Google Play Music. I normally have many more tabs open, but these three I access periodically. As the number of open tabs increases, and I fail to cleanup, getting back to my usual three gets a little trickier.

So far, I've kept each of these services open in separate windows. But that doesn't always work. Click on any link in your inbox and you're back to playing cleanup. This is where site-specific browsers (SSB) can help.

Epichrome enables you to build Chrome-based SSBs (on Mac OSX). Build an SSB for the usual suspects and easily jump to them using the app switcher.

More at OSX Chrome SSB and Quora thread

UPDATE: Enable the Chrome extension to open URLs in default browser.

And choose the default browser to open URLs.

Type. Publish. Done.

Josh Thompson — Wed, 22 Jul 2015 03:00:00 GMT

Yesterday I read How the Hell do I Prioritize Work, Blog & Find Balance.

The author of the letter is a busy, accomplished guy and still manages to write regularly.

He said, in short:

I sit down, and I write. I’ve done it a lot, so I’m not bad at it. I don’t often proof read my stuff, I make grammar and spelling errors, and what I publish is not polished, refined.

I’ve been doing it for long enough that I can crank out posts quickly. I don’t worry about typos. I don’t worry about sounding too smart. I don’t worry about every post becoming part of the national treasures. I just write. Usually single sitting. No editing. Hit publish. Sayonara

I like that style. “Single Take”. Spill what’s on your mind and hit publish.

I get caught up in editing, making things be just right, putting out “amazing content” that I just write literally nothing. Nothing at all.

There are times and places for great content, but this is not it. I don’t need a great banner image. (Cropped to look great full-width, of course, and something with text over it to look good for social media sites…)

Is money changing hands? (Or are you hoping to convince someone to buy something?) Great, do a round of editing. Make sure that what you are talking about is resonating. But this is not that.

A Quick Look into Stamp Duties & Land Taxes

Luke Singham — Wed, 22 Jul 2015 03:00:00 GMT

Stamp Duties #

Stamp duties are a transaction tax. Prior to the digital era, the transfer of legal ownership presented a convenient time for governments to levy a tax. Upon the transfer of ownership a stamp duty was paid and the relevant transfer of ownership document was either stamped...

Or a stamp was affixed...

Nowadays, most Australians recognise stamp duties as the tax paid upon the purchase of a property. For example, in NSW the rate of stamp duty ranges from 1.25% to 7%.

Source: osr.nsw.gov.au

Land Value Tax #

A popular alternative, or at least with economists, is that of a Land Value Tax (LVT). Despite the idea having been argued for since Adam Smith, the idea is most strongly associated with Henry George who famously argued in his best selling book Progress and Poverty for an LVT to replace all other inefficient taxes.

The crux of the argument is that since land is fixed (as opposed to say the supply of labour or level of consumption) disincentives or distortionary effects of land taxes are not applicable. On the other hand, a stamp duty causes a distortion to the property market by disincentivising transactions. Logically, stamp duties decrease the propensity of someone to move, thus reducing labour mobility. Those individuals who have to relocate often pay more in stamp duties, which unless there was some reason to discourage people from moving, is inequitable. The Treasury's graphs below show the shape of the effective tax rate of a stamp duty over years of occupancy. The panel B graph shows the impacts of several moves resulting in a much higher effective tax rate. A flat rate shows the effect of a land tax (flat rate).

By replacing stamp duties with an annually payable LVT land hoarding behaviour would be heavily discouraged. Governments would stand to benefit as stamp duty revenue fluctuates with the property market, whereas a land value tax would provide a steady annual revenue changing only when land values are revised.

The Henry Review makes several important points in considering a land tax:

A land tax could encompass all property types such that the use of land is more likely to go that which is most productive, whether that be commercial, residential or industrial.
Tax-free thresholds could apply to exempt low value land. Particularly of benefit to agricultural land holders.
Different rates of land tax could be constructed in a progressive manner. Much like the increasing rates of income tax or stamp duty seen above.
Replacing stamp duty with land tax would remove the supply impediments caused by the tax system. However, the tax system itself is not the major source of supply constraints.
Currently land tax policy exempts owner-occupied housing, removing 60% of land by value from the potential revenue base.
To avoid double taxation of those who have already paid stamp duty, the tax could be introduced by only applying to new transfers.

The Grattan Institute estimated in 2012 that replacing stamp duty with a land tax would increase GDP by $25 billion. Other research based on Melbourne house prices suggested that the introduction of a land tax would decrease average house prices by 5% and would have a greater reduction on inner CDB house prices with up to 12% reductions.

One potential issue with the introduction of the policy is the impact it would have on the asset rich but income poor i.e. pensioners. However, as Michael Janda points out:

... it would be very simple for state governments to allow such people to defer their land tax payments, which would be indexed at a suitable rate and only fall due when the property next changed hands. Thus the land tax bill could be automatically deducted from the sale proceeds of the property when it was eventually sold.

Another administrative consideration is the difficulty in valuing land. However as the Economist points out, "the efficiency of the tax does not depend on accurate valuations."

The biggest issue is political...

People don't like paying taxes. The issue with a land tax is that it is hard to avoid and an annually payable (or annually painful). It would be very easy to politically tar such a economically sound policy, as the great big new tax grab. This hasn't stopped up to twenty countries having implemented or are considering implementing the policy at some level.

In Australia, the ACT introduced a land tax three years ago to replace stamp duties. This year, the South Australian government has removed stamp duties on business transfers and will remove stamp duties on commercial property transfers in 2016. However, they have backed down on their suggestion earlier this year of replacing stamp duties with a land tax on residential properties for political reasons.

There are of course bodies such as the Urban Development Institute of Australia (UDIA) and the Property Council who would stand to gain from a tax change that would increase property market activity. Active engagement of such bodies along with public education would be necessary for a government wanting to minimise political fallout.

For now it remains the economist's consensus and the politician's discord.

Generating Read Models with Event Sourcing

Daniel Little Dev — Wed, 22 Jul 2015 01:28:00 GMT

Event sourcing is a very powerful tool that allows you to capture information you didn't even know you wanted. One of the things you usually…

A guide to PR for startups

CRAIG KERSTIENS — Tue, 21 Jul 2015 23:55:56 GMT

You’ve built your product and you’re now ready for your first major launch. Or you’ve been through a launch or two, but are looking to scale the process as you’re doing more launches and announcements. You really have two options: do it all on your own, or work with a PR agency. One frequent crossroad is that you’re not at the point of a full time PR person, but unsure what a PR agency can offer you; and, further what’s the best way to work with them so you’re getting the maximum value.

As I’ve talked to more startups lately, it’s become clear that effectively working with PR teams and the media is mostly learned by doing. Because there’s not much guidance out there, here’s an attempt at some basic guidelines.

On PR

First there’s two types here and they’re not mutually exclusive. In-house PR is a full time person or team that works within your company, here you’ll often have a pretty different experience. From my experience, in-house PR people tend to understand a company message and vision because they are living and breathing your company values every day.

The other alternative is hiring a PR agency. An agency will have several (sometimes hundreds!) of clients. The relationship that you’ll have with an agency is much different than in-house. You’ll use them just like you would a consultant or contractor. Most startups end up with the agency approach first, because of the perception of “more people working for a cheaper cost than hiring in-house.” However, it’s of note an agency doesn’t alleviate you of doing work, nor should you want them to handle all parts of it.

Messaging

An agency may offer to help with messaging, but take this somewhat lightly. I don’t doubt that some are very good at it, but in most cases I’ve found they don’t have the same amount of customer interaction as you as a founder or early employee would. Further, your vision of impact to the market and direction may be more distant than theirs. You should expect to own your messaging, just like you own your product.

Where they can heavily help is providing a lot of structured frameworks for helping you get to your messaging. Some pretty basic templates of standard questions for customers and partners can go along way in helping you actually uncover what they feel your value is.

On your key messaging/value prop, there’s two pieces I’ll drop in here. While I’d love to write another long post on it, I wonder when I’ll actually get it out. So the first is pitch the problem you’re trying to solve–Dave McClure talks about this as well as anyone. The second is don’t pitch features, pitch the use cases and solutions. Pitch what’s possible

Pitching

This is the number one area I’ve found that having PR makes a huge difference. In the world of reporting, different reporters have different beats (areas of coverage), styles, outreach preferences, and most importantly, different relationships with companies and people. Knowing all of this and how to pitch a story to them is key. Yes you can spend hours researching and creating a perfect story just for them, and do that again, and again and hopefully land some coverage. But I’d argue a bit: that’s not the best use of your time.

With a good PR person or agency you’ll be able to strike a mix of:

Here’s the outlets I want to be in and why (have a good reason for why).
Understanding the audience and readership.
What outlets you feel like your key customers are reading, and validate this with the agency.

From there, if you’ve found a good agency they already have relationships with your key journalists / publications. So if you have a compelling product, you just need to give them the right messaging of the particular launch or news.

What else to expect from your agency

A surprise for some is how the whole process works. The agency is going to be there on the phone with you. You’re not going to hang out over beers while pitching being chummy. The reporter is listening to multiple other pitches, it’s likely they had one right before you and right after. The agency is there listening, helping keep time and track of conversation for reporter fact-checking after the interview.

Hopefully they’re also keeping notes. They should be able to provide you with some high level notes of what message resonated with each reporter and what didn’t, what you covered, and what they asked. This is especially useful for future interactions.

Similarly you should get a briefing 1 pager ahead of time. You should be able to skim this, you don’t have to memorize. But it’ll include key things about recent articles written by the reporter, their beat, topics to dive into and ones to stay away from. If you can connect the dots, those notes from an initial call start to feed into the 1 pagers for future calls.

Onto the briefing

Of course it’s important to land the briefing in the first place, but just as important is getting it right. Coming into it, the reporter will have already gotten the high level pitch… It’s why they took the call. You’ll get a mixed bag of those that are open to teeing up the opportunity to those that want to get right to the news. Roll with what they prefer, but also don’t be afraid of trying to hit some of your key points.

Have your key messages ready

Sound bites help hugely here. Analogies, customer references, whatever you want to hit. Have it ready. Also if you’ve got a great sound bite that helps tell the story, it can make the reporter’s job easier. Just don’t swing too far into happy go lucky marketing land. It’s important to remember that you’re talking to a person. Have a conversation - don’t talk at them.

Go slow

It may seem obvious when you think about it, but as you’re talking the reporter is writing. Or at least you hope they are. Some do it by hand and type up notes late, some type right then and there. When you hear a pause it doesn’t always mean to keep going and it seldom means hurry up. Become extra comfortable with pauses. Check in if you’re going to fast, if they’re following, if they have any questions. I’ve had people bring me in a beer before because I’d had multiple cups of coffee through a few pitches, and they were trying to slow me down a bit. Know your pace, and then slow it down.

Questions

It’s okay if they don’t have a lot of questions, they may not. They may have none at all. Yes, pause, and give them a chance, or even ask if they have any. But don’t stress too much if they have no questions.

On the flip side of that - you’re PR person should have prepared a list of questions for you beforehand that the reporter could possibly throw your way. Be sure you’ve thought through and practiced all the Q&A scenarios before the interview so you aren’t caught off-guard when you’re in front of the reporter.

In conclusion

If it’s your first go around, don’t stress too much. Have the headlines you want in your mind and key messages, or better yet write them out. Personally I write key things on a whiteboard nice and large before I’m on the call. Finally once you’re all done, enjoy reading the coverage. But you’re not all done after you get some coverage look back, run a retrospective just like you would for a software project. What worked well, why did or didn’t something work. What can you improve next time.

*Full disclosure, this is based across interactions with a small sample size of different PR agencies and individuals. Mileage may differ heavily from PR firm to PR firm, but hopefully the above provides at least some roadmap for more clarity vs. flying blind. As always if you’ve got feedback/questions, feel free to let me know @craigkerstiens

Finally a special thanks to Paul Katsen for much of the inspiration on creating this post and to he and Katie Boysen for review

Five ways to avoid and control flaky tests

Ben E. C. Boyter — Tue, 21 Jul 2015 11:09:42 GMT

Having a reliable test suite should always be the goal in software development. After all if you can’t trust the tests then why bother running them at all? This is especially important in a shared coding environment and when running through Continuous Integration (CI).

1 - Test in Isolation

It may seem obvious but writing focused tests which do a single thing is one of the most effective ways to avoid them being flaky. Tests which do multiple things increases the chance for failure and can make the tests non deterministic. Always remember to test features and issues in isolation.

Delayed Serial STONITH

smcleod.net — Tue, 21 Jul 2015 03:00:00 GMT

A modified version of John Sutton’s rcd_serial cable coupled with our Supermicro reset switch hijacker:

This works with the rcd_serial fence agent plugin.

Reasons rcd_serial makes for a very good STONITH mechanism:

It has no dependency on power state.
It has no dependency on network state.
It has no dependency on node operational state.
It has no dependency on external hardware.
It costs less that $5 + time to build.
It is incredibly simple and reliable.

Essentially the most common STONITH agent type in use is probably those that control UPS / PDUs, while this sounds like a good idea in theory there are a number of issues with relying on a UPS / PDU:

Why Does Software Contain Bugs?

Ben E. C. Boyter — Mon, 20 Jul 2015 11:06:50 GMT

“Why does all software contain bugs?” this was a question recently asked of me. My response at the time was because all software is not perfect, but is this true?

Lets take a very simple example.

    public class Hello {
        public static void main(String[] args) {
            System.out.println("Hello World!");
        }
    }

The above example is arguably the simplest program that can be written using Java. It also happens to be the first program usually written by any Java programmer. It simply print outs the text “Hello World!” when it is run. Surely this program is so simple that it is perfect and therefore bug free?

No such thing as "real programming"

Brightball Articles — Mon, 20 Jul 2015 04:34:00 GMT

I read an article earlier today called The self-hating Web Developer that I found on Hacker News and it bothered me. It resonated with me as something that I professionally internalized over my career but it bothered me because Joseph encountered personal financial difficulty for both himself and his family due to the struggle. For that reason (and insomnia), I feel compelled to write this as reference to refer to for anybody else who may be struggling with the same thing.

Sardinia travel bookmarks

xenodium.com @alvaro — Mon, 20 Jul 2015 03:00:00 GMT

Alghero.
Baja Sardinia.
Budoni.
Cala Goloritze, Sardinia.
Castelsardo (gifts maybe?).
Food: Maialetto sardo (Pig), Sebadas, Pardula, Papassinas, Pani e sapa.
L'Asinara boat trip (abandoned penitentiary).
La Pelosa beach.
Nuraghe.
Porto Cervo.
Porto Torres.
San Teodoro.
Stintino (fishing port).
Zedda e Piras vinyards (Alghero).

Vagrant and Ansible

A Holding Company — Mon, 20 Jul 2015 03:00:00 GMT

I’m learning Python, and have had fun automating tasks with Python scripts.

I was polluting my Macbook (Yosemite) quite a bit by installing all libraries globally, until I learned about Vagrant. Vagrant is essentially a command-line wrapper around VirtualBox. It makes it easy to create/configure/destroy virtual machines from the shell.

Vagrantfile ¶

I installed it from here. A summary of my journey is below.

Created and jumped into a new directory:

$ mkdir newproject && cd newproject

Create a Vagrantfile. This creates a file named “Vagrantfile” and a “.vagrant folder” in the directory:

HillHacks 2015

Nemo's Home — Mon, 20 Jul 2015 03:00:00 GMT

A little while back, I came across HillHacks, a conference in Dharamshala about “hacking and making in the Himalayas”. I was instantly hooked. It took a lot of scheduling troubles, but I decided to stay for the entire unconference, which started at 23rd May.

Its hard to describe the HillHacks experience in a single blog post. I met so many amazing people from all over the world. Learned a lot of different things. I Had a lot of fun teaching some other things. I helped organize some of the stuff, and managed to stay awake an entire night while participating in an CTF. And on top of that, got to eat delicious food.

HillHacks as an event, was divided into two segments:

An unconference (23rd May - 3rd June)
Main Conference (4-7 June)

A lot of people had arrived before me at the venue and taken care of the basic infrastructure. We had internet connectivity via two local ISPs. We had IPV6 connectivity via a tunnel in Belgium as well.

There were a lot of fun activities planned everyday: from unicycling to skateboarding and playing Cards against Humanity; it was a lot of fun living with so many strangers and trying to figure out ways to help.

I did a talk on SDSLabs, a quiz for everyone, and an introductory session on CTF contests. We then participated in a CTF organized in Germany as Team HillHacks. On the last day of the conference, I did a [talk][josd-talk] on “The Joy of Software Development”, which is a book I am working on.

For the first time in my life, I met people who actually use BSD. And to make it even more amazing, I met NetBSD Kernel developers, people on the BSD Security Team, and people who prefer OpenBSD over NetBSD (I’d never really cared for the distinction, as a Linux user)

We did a lot of hacks, including running an MPD Daemon and streaming it over IceCast. I also spent a lot of time cubing and teaching people how to solve Rubik Cubes. My times have also improved somewhat as a result. Thanks to trouble, I also learnt how to solve a MegaMinx.

As part of the School Outreach program (organized by the brilliant Tink), we taught kids about Codes and Ciphers, programming, speedcubing and lots of other things. The kids also performed in the final Gala Show giving us brilliant performances in 3 different plays (all 3 schools had their own plays).

I learned a lot of different things: how to start with Kernel Programming, DNSSEC, Retro Gaming. Thanks to a few dedicated volunteers, we even made a 8-inch Telescope that made staring at the night sky so much fun. We had a session on Typography, a story telling session in Malayalam (translated to English on the fly). I even learnt a bit of Emacs.

The list is so long, I don’t think I can do it justice in this single blog post.

The most amazing part was not the technical things, but the community itself. sva would often say that everyone of us has “sudo access on the conference” (geekspeak for full authority). Each of us helped organize it, any way we could. The community got together to setup the stage, tents, network and the entire infrastructure at HillHacks. Zainab even has a blog post on social cooking at Hillhacks.

As I sit here at the venue, it has been 2 weeks of fun and awesomeness here at HillHacks. I leave with lots of memories and hope to be here next year.

If this blog post interests you, be sure to check out hackbeach as well. We are doing a mini-conference around November in Kovalam.

Network sockets—an ordeal (starring C++11)

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Sun, 19 Jul 2015 22:59:12 GMT

Inspired by Ann Harter’s post about three dead protocols, which I would not pronounce dead just yet, I also decided that it was time to re-connect with socket programming. I foolishly decided that this was also a great time to apply some of the new features of C++11. Kill two birds with one stone, or something like that. In the end, I narrowly escaped with my sanity, but was able to finish at least an implementation of RFC 865, the Quote of the Day protocol.

The quest begins

C++ does not have a standardized API to deal with sockets. It probably never will. Although there are some higher-level implementations out there, such as Boost.Asio, my inner Klingon decided that it would be more honourable to learn the concepts from the venerable POSIX sockets API. To a C++ programmer, this API is somewhat odd, to say the least. Lots of usage of void* force the diligent programmer to make keen use of reinterpret_cast. Functions return an integer, with -1 indicating an error, and the magical errno variable describing the cause.

Perusal of the documentation yielded the following information for creating a server socket that is reachable via TCP/IP:

Creating a socket with domain AF_INET and type SOCK_STREAM.
Setting the socket option SO_REUSEADDR of level SOL_SOCKET. Otherwise, stale connections might hinder the program from re-binding to the desired address after quitting and restarting the server.
Creating an instance of sockaddr_in and filling it with details about the desired address and port to bind to.
Calling bind() to perform the actual binding.
Calling listen() on the socket to wait for connections. But wait, this does not actually do anything! To really "wait" for a connection, we have to accept() it first.
The call to accept() is blocking by default, meaning that program execution will stop until there is some client to be accepted. The call then returns a socket describing the client connection.
The returned socket is a socket which we may finally use to send some data to!

I wrapped the calls described above in a slightly-improved interface for a simple server class so that other programmers do not have to deal with the innards of the POSIX sockets API as much. This is what the API looks like so far:

Server server;
server.setBacklog( 10 );
server.setBacklog( 2015 );
server.listen();

It is almost-but-not-quite usable. However, we only arrived at obtaining a socket for a client. The socket is again an integer that requires the use of the sockets API. Feeling more foolishness rise in me, I hastened onwards and wrote a nice class for wrapping a client socket. At present, it only wraps the send() function of the POSIX sockets API. But at least I am now able to send strings instead of const void*, hooray:

ClientSocket socket( fd ); // fd is coming from somewhere else; see below...
socket.write( "So much wow!" );
socket.close();            // Close the connection after so much excitement

Enter C++11

Until now, C++11 did not play a large role. Since my pretty server API only reacts to new client connections (it cannot know whether a client actually sent something), I though it would be nice to have a user-configurable functor that is called asynchronously whenever that happens. Enter std::function and std::async. The user needs to specify a function for handling accepted connections:

server.onAccept( [&] ( std::unique_ptr<ClientSocket> socket )
{
  socket->write( "Your quote here, for only 9.99 USD!" ); 
  socket->close();
} );

The server then launches this function asynchronously whenever an accept() call returns:

auto clientSocket = std::unique_ptr<ClientSocket>( new ClientSocket( clientFileDescriptor ) );
auto result       = std::async( std::launch::async, _handleAccept, std::move( clientSocket ) );

I am using an std::unique_ptr because the server does not want to take ownership of the client file descriptor at this time. Maybe I am going to change this in the future.

So?

If this seems like a lot of hassle for doing something as simple as sending a random quote to a client, you are right. I cried hot tears of shame when I compared my 203 lines of code to Ann’s 37, which even contained some commented code:

require 'socket'
require 'csv'

quotes_array_unparsed = CSV.read('goodreads_quotes_export.csv')
keys = quotes_array_unparsed.delete_at(0)

count = 0
quotes_array = []
while quotes_array.length < quotes_array_unparsed.length
  quotes_array_unparsed.each do |quote|
    quotes_array[count] = Hash[keys.zip quote]
    count += 1
  end
end

quotes_array.each do |hash|
  hash.each do |key, value|
    value.gsub!("<br/>", "\n")
  end
end

#def less_than_512
#  if @quote_body.bytesize < 512
#	qotd(@quote_body, @quote_author)
#  else 
#    less_than_512
#	end
#end

server = TCPServer.new 17

loop do
	Thread.start(server.accept) do |client|
		random_index = rand(quotes_array.length)
		@quote_body = quotes_array[random_index]["Quote"]
		@quote_author = quotes_array[random_index]["Author"]

		def qotd(quote, author)
			"#{quote}\n   - #{author}"
		end
		client.puts qotd(@quote_body, @quote_author) 
		client.close
  end
end

I am amazed! Of course, this comparison is slightly unfair because I had to write my own version of Ruby’s TCPServer module. Still, this code is definitely more elegant than mine. To compensate for this, my implementation of the Quote of the Day protocol serves up random quotes from Ambrose Bierce’s The Devil’s Dictionary, which I hope will get me some pity points.

Where is the code?

Please find the code on its GitHub repository. I plan on doing at least an implementation of RFC 862, the Echo protocol as well, but this will require more changes to the client socket, namely the ability to read stuff as well.

The code is released under an MIT licence.

3-way Disk Mirrors With ZFSOnLinux

BHoey — Sun, 19 Jul 2015 05:03:22 GMT

Traditionally mirrored disk sets in Linux and other operating systems have been limited to two devices. While mirroring has the benefit over other raid levels in that each mirrored device contains a complete copy of the data, the two device limit became inadequate as disk sizes ballooned. In the age of multi-TB drives, simply rebuilding a degraded mirrored array could actually cause the surviving device to fail, eliminating the very redundancy one was expecting.

ZFS addresses this particular problem in several ways through data checksums, self-healing and smart resilvering instead of blindly rebuilding full array members even if only 1% of disk space is being used. [Continue reading...] 3-way Disk Mirrors With ZFSOnLinux

The post 3-way Disk Mirrors With ZFSOnLinux appeared first on BHoey.

Maybe "Now" Is Not the Right Time

Josh Thompson — Sun, 19 Jul 2015 03:00:00 GMT

Recently I deleted a bunch of old notes I had in Evernote. Some of the notes were almost immediately unneeded, like old receipts and confirmations.

Much of the rest was notes related to goals (“Checklist to move out of MD Apartment”, “Planning trip to Buenos Aires”) or to projects I had. (“Set up personal website”, “Outline book on overcoming fear of lead climbing”, etc).

Some of the things I’ve been working on in the last few weeks I’ve been thinking about for over a year. Whoa. Why did I wait until now to do something that I’d obviously wanted to do over a year ago?

“Now” is a good time to do something, but “later” isn’t always bad.

I’ve tried to find a different way of saying this, but anything else isn’t honest. I’ve felt tremendous guilt lately at not accomplishing some of the projects I’ve laid out for myself.

I’ve not studied any Spanish, even though I’ve been living in Buenos Aires, fumbling around with my terrible Spanish skills.
I’ve not taken measurable steps on either of two side projects I’ve been meaning to accomplish (and have been wanting to do them for years.)
I’ve spent time reading mostly pointless fiction rather than more meaty non-fiction/histories/biographies, even though I’d much rather spend time in the latter.
I’ve only in the last few days feel like I’ve made significant progress with projects at work.
I’ve not done a single physically challenging act in two months. No climbing, no exercise.

All of these things will be accomplished in time. While I shouldn’t sit around until I magically feel motivated to do all of the undone things in my life, I can respect the ebb and flow of motivation, and peg the right projects to the right time.

I do this at a daily and weekly level (I.E. certain kinds of work when I’m feeling high/low energy, and I keep work limited to mostly-normal working hours), so it is no stretch to extend the principle to a monthly/yearly schedule. (Except for the exercise bit - I feel way better with exercise in my life, and allowed the current inaccessibility of climbing to derail that habit.)

It’s hard for me to not succumb to the Instagram/Hacker News effect. (I just made that up. It’s the endless viewing of other people’s beautiful lives/successful products makes me think that I am a dismal failure, just because I’m not currently launching something elegent/successful/lucrative/world-changing or having a fabulous time in a beautiful place.)

How does this matter to you?

The same way that it matters to me. Do things that matter. But don’t lose sleep over not doing everything as quickly as you originally thought you might, and don’t be beholden to doing things that you no longer need to do.

It’s the slope, not the Y-intercept.

Age of Sigmar breaks Warhammer Fantasy, but it's worth it

Ben Overmyer's Site — Sun, 19 Jul 2015 03:00:00 GMT

Age of Sigmar grabbed the Warhammer Fantasy rulebook, tore it apart, and threw the remains into the fire. I love the game that came out of that, even if there are some significant negatives to this event.

From simulationist roots to casual narratives

Warhammer Fantasy has long been the strategy-heavy cousin to the tactical-heavy Warhammer 40,000. The rules up until this point focused on maneuver and regiments, positioning and thinking ahead. Warhammer 40,000, on the other hand, has always been about skirmishes and characterful, tight battles. Where Fantasy relied on its movement trays and slower pace, 40k allowed for a lot of narrative to come through the gameplay. Age of Sigmar one-ups 40k in this regard. Yes, Age of Sigmar has a lot of holes in the rules. Someone who delights in using rules-as-written against his opponent is going to have a field day with Sigmar. But that's not what this iteration of the game is about. Age of Sigmar seeks to bring casual gaming to the fore. The rules explicitly encourage, via its Most Important Rule, house-ruling and collaborating with your opponent to set up the battle.

Play any army you come up with

One of my favorite things about the rules is its encouragement of creativity in army building. There are no points or required force organization. While this can obviously be abused by asocial types, for those of us that enjoy narrative battles, it's a godsend. I can mix Seraphons (Lizardmen) in with Stormcast Eternals and Exile Aelfs (Dark Elves), and that's perfectly legitimate. If I really wanted to, the rules allow me to field Chaos Warriors alongside Sylvaneth (Dryads) and Duardin (Dwarfs). I can't imagine the scenario where that would make lore sense, but you could do it. All of this makes for some fantastic battle and lore possibilities.

My God, it's beautiful

The miniatures in the starter box set are absolutely gorgeous, and make the collector in me very happy. The Lord Celestant in particular is a beautiful model. With his triumphant pose and intimidating Drakoth mount, he makes a fine addition to any army. The Khorgorath model on the Chaos side is really ****ing creepy. It's made of blood and skulls of its victims are pushing their way through its skin.

There's a free app for army building

This is a big one. I bought Army Builder ages ago to help me build 40k armies, but Games Workshop is notorious for never releasing anything for free. Privateer Press released a free app awhile ago for its Warmachine and Hordes games, and while you need to pay for your specific army, it does have the benefit of being a first-party app. Now, GW has released the Age of Sigmar app that includes an army builder and the Warscrolls for all existing models for free. The one thing that pisses me off about this app is that I don't get the Warscrolls for the starter box units, despite having shelled out $125 for said box. There's no way to redeem a product code or anything. The only way to get those Warscrolls in the app is to buy the Age of Sigmar book through it, which would be an additional $50. No thanks. But it's GW, so I'll give them points for the progress they did make.

...tournaments?

Yeah, the Age of Sigmar rules pretty much break any possibility of having tournaments without extensive house-ruling. That's a problem for people who're really into the game. I'll admit to being a little hesitant about this. I've never played in a 40k tournament, but I think it'd be fun to try. I can't imagine playing in an Age of Sigmar tournament, though. The rules aren't geared for that.

What about the people who enjoyed the old game?

Games Workshop's move to completely get rid of the old rules and cease supporting it in their stores is a bad one, despite the good game that Age of Sigmar is. The community that built up around WHFB is not so easily dismissed. For some people, their local Games Workshop store is the only place they could play the game they'd put a lot of time and money into, and I've heard rumors that GW stores are disallowing WHFB matches now. That's a terrible policy. Mantic Games has their Kings of War game, which keeps very closely to WHFB's roots. They just released a second edition, and have been inviting WHFB players left out in the cold to join their community. This is a great move on Mantic's part, and I really hope it works out for everyone. From what I've seen, Mantic cares a lot more about community than GW does.

So is it worth getting?

Yes. Especially since the rules are free, and the Warscrolls - the rules for individual units - are also free. You can get them on Games Workshop's site. In my opinion, the starter box is also totally worth the $125. It has wonderful minis, and the books are great too.

Hadoop on Ceph: usability survey

Noah Watkins — Sun, 19 Jul 2015 03:00:00 GMT

Hadoop interoperability with Lustre, Quantcast, Gluster, and more.

Jobs

🦉 olshansky 🦁 — Sat, 18 Jul 2015 11:57:37 GMT

Olshansky's review of Jobs

Get Hard

🦉 olshansky 🦁 — Sat, 18 Jul 2015 11:56:21 GMT

Olshansky's review of Get Hard

The Effective Engineer

june.kim — Sat, 18 Jul 2015 03:00:00 GMT

Doors of Italy

Sam Solomon — Sat, 18 Jul 2015 03:00:00 GMT

My trip to Italy last year was the first time I’d ever been overseas. Most of the trip was spent walking across Tuscany and Umbria.

Migrated away from Heroku

Ben Overmyer's Site — Sat, 18 Jul 2015 03:00:00 GMT

Heroku made their free tier awful. That's fair enough, but I need more stability.

So, I've spun up a Digital Ocean droplet and learned how to deploy Rails apps without the help of a PaaS.

It's not intuitive for someone who's used to provisioning PHP web app servers, but after finally getting it working, it's actually a pretty cool way of doing things.

So yeah. Obelisk, the Ruby on Rails blog engine I've written to replace the PHP-based Monkblog engine, is now running on a server that I've provisioned from scratch. Yay!

searchcode the path to profitability

Ben E. C. Boyter — Fri, 17 Jul 2015 11:01:15 GMT

One of the things that has always bothered me about searchcode.com was that it never generated any money. Not a huge problem in itself as a side project, but the costs to run it are not insignificant due to the server requirements. I had looked into soliciting donations but I considered this highly unlikely to produce enough revenue to cover costs considering that sites such as gwern.net was unable to make enough to cover even basic costs through patreon (although since a recent HN post this has jumped from around $20 a month to over $150).

Open closest build file in Emacs

xenodium.com @alvaro — Fri, 17 Jul 2015 03:00:00 GMT

Whether it's Makefile, SConstruct, BUILD, or your favorite build file, chances are you have to tweak it from time to time. ar/open-build-file searches your current and parent directories to find a build file.

(defvar ar/project-file-names '("Makefile" "SConstruct" "BUILD"))

(defun ar/parent-directory (path)
  "Get parent directory for PATH."
  (unless (equal "/" path)
    (file-name-directory (directory-file-name path))))

(defun ar/find-upwards (path filename)
  "Search upwards from PATH for a file named FILENAME."
  (let ((file (concat path filename))
        (parent (ar/parent-directory (expand-file-name path))))
    (if (file-exists-p file)
        file
      (when parent
        (ar/find-upwards parent filename)))))

(defun ar/open-closest (filename)
  "Open the closest FILENAME in current or parent dirs (handy for finding Makefiles)."
  (let ((closest-file-path (ar/find-upwards (buffer-file-name)
                                                 filename)))
    (when closest-file-path
      (message closest-file-path)
      (switch-to-buffer (find-file-noselect closest-file-path)))
    closest-file-path))

(defun ar/open-build-file ()
  "Open the closest project file in current or parent directory.
For example: Makefile, SConstruct, BUILD, etc.
Append `ar/project-file-names' to search for other file names."
  (interactive)
  (catch 'found
    (mapc (lambda (filename)
            (when (ar/open-closest filename)
              (throw 'found t)))
          ar/project-file-names)
    (error "No project file found")))

A Quick Guide to Net Promoter Score

Sam Solomon — Fri, 17 Jul 2015 03:00:00 GMT

Looking for a quick guide to Net Promoter Score? You’ve found it. This little guide will give you all the basics.

angular-fng

Writing on Adam Craven — Fri, 17 Jul 2015 03:00:00 GMT

The benefit of testing for Developers, Managers and the Business

Ben E. C. Boyter — Thu, 16 Jul 2015 11:05:30 GMT

“Fixing regression bugs is analogous to digging a hole only to find the next day it has been filled in and having to dig it out again”

Ask any manager, developer or tester working on software without tests what the main pain points are. Nearly all the time the main one mentioned is dealing with regressions. This is bugs that were fixed a year ago which returned. Regression bugs cost the software industry billions of dollars a year. Worse still they are demoralising to everyone involved. Finding or fixing the same bug over and over causes you to start looking for new projects or new jobs.

Create iOS static fat libraries

xenodium.com @alvaro — Wed, 15 Jul 2015 03:00:00 GMT

Have separate static libraries for different iOS architectures? Stitch 'em up into a single fat library using with lipo:

$ lipo -create libOne_i386.a libOne_x86_64.a libOne_armv7.a libOne_arm64.a -output libOne.a

Settling scores with an org table

xenodium.com @alvaro — Wed, 15 Jul 2015 03:00:00 GMT

Recently kept track of expenses between a group of us. To settle the scores, I emailed an exported HTML table from an org file. This was simple enough and required no external viewer from recepients. The org table, in all its textful glory, looked as follows…


| Date             | Item           |   Charge |
|------------------+----------------+----------|
| [2015-06-18 Thu] | Cash           |    20.00 |
| [2015-07-11 Sat] | Lucky 7        |    42.97 |
| [2015-07-13 Mon] | Santa Maria    |    32.00 |
| [2015-07-12 Sun] | Tayyabs        |    46.00 |
| [2015-07-13 Mon] | The Brass Rail |    39.00 |
| [2015-07-13 Mon] | Underground    |    10.00 |
| [2015-07-10 Fri] | Cash           |    20.00 |
| [2015-07-13 Mon] | Cash           |    20.00 |
| [2015-07-14 Tue] | Cash           |    20.00 |
|------------------+----------------+----------|
|                  | total          | £ 249.97 |
#+TBLFM: @11$3=vsum(@2..@10);£ %.2f

…while the exported HTML below could be easily pasted on to an email.

Date Item Charge

[2015-06-18 Thu] Cash 20.00 [2015-07-11 Sat] Lucky 7 42.97 [2015-07-13 Mon] Santa Maria 32.00 [2015-07-12 Sun] Tayyabs 46.00 [2015-07-13 Mon] The Brass Rail 39.00 [2015-07-13 Mon] Underground 10.00 [2015-07-10 Fri] Cash 20.00 [2015-07-13 Mon] Cash 20.00 [2015-07-14 Tue] Cash 20.00 total £ 249.97

#+TBLFM: @11$3=vsum(@2..@10);£ %.2f

Battlehack Athens 2015

nutcroft — Wed, 15 Jul 2015 03:00:00 GMT

This May I attended for the first time the Battlehack hackathon series. It was also the first time they were coming to Athens.

We implemented an Ionic + Sails.js app, which enables you to help people of your local community. The idea is to learn and teach people stuff, either for profit or voluntarily. The only prerequisite to participate for the winning prize was to use either PayPal or Braintree for payments, so we just used it for those who wanted to pay/get paid in the app.

Overall, it was an awesome experience. Braintree (the company behind the organization) had prepared everything. Tons of food present at every moment, which does not mean that there weren't also separate meals; coffee and red bulls, even massage sessions!

After the hacking ended we had to present our creation. Although our app decided not to behave for these crucial 2 minutes of our pitch, some people, i.e. sponsors, liked us and gave us two gifts for using their APIs.

Definitely recommended as an experience, it was very fun and I already plan to attend next year. The only regret was not sleeping at all the previous day, which resulted in ~60 hours of sleeplessness; at the bucket list of not to do again!

P.S. Highlights video.

Walden

june.kim — Tue, 14 Jul 2015 03:00:00 GMT

FatturaElettronicaPA for .NET has been updated

Nicola Iarocci — Tue, 14 Jul 2015 03:00:00 GMT

FatturaElettronicaPA has just been updated to v0.1.4. With this release invoice bodies (FatturaElettronicaBody items) are also validated. As always, you can install the package directly from NuGet. See the original post for more info. Also don’t forget to check the related projects.

Update: v0.1.6 has also been released.

Recognize new password prompts in Emacs shell

xenodium.com @alvaro — Mon, 13 Jul 2015 03:00:00 GMT

At some point, you may come across a trusted command-line utility prompting you for a password, and Emacs shell happily displaying each typed character to the nearby-world to see. Luckily, you can train Emacs to recognize new password prompts and hide the typed characters in modes deriving from comint. Append the password prompt REGEXP:

(setq comint-password-prompt-regexp (concat comint-password-prompt-regexp
                                            "\\|"
                                            "Password for red alert:"))

[Review] "Data and Goliath" by Bruce Schneier

ntietz.com blog - technically a blog — Mon, 13 Jul 2015 03:00:00 GMT

I just finished reading Bruce Schneier's latest book, "Data and Goliath." I was apprehensive at first -- I'm a big fan of Schneier's posts online, but I found this randomly at the library and I was hoping not to be disappointed. In the end, it was well worth the read.

The book was split into three parts. In Part One, he discusses what a world of constant mass surveillance looks like. He illustrates what data everyone is leaking through ordinary activities, how people can and are monitored, and how this data can be used. In Part Two, he explains what is at stake: what the political and economic losses of surveillance are both in the US and abroad. And in Part Three, he explains what can be done about this in a three prong fashion: what the government should do; what corporations should do; and what we, the people, should do. All throughout, he provided compelling examples and illustrations, as well as footnotes with additional references (although, confusingly, these are not referenced inline but are merely listed at the end).

There were many compelling points in this book, and I can't list them here, but I want to call attention to one in particular. He puts out a call to action for the tech community to (paradoxically) create surveillance tools for the government to use - the argument being that "if we want organizations like the NSA to protect our privacy, we're going to have to give them new ways to perform their intelligence jobs".

Overall, I think he did a great job making these issues available for a non-technical audience. It was written in a way that will be open to everyone inside or outside the tech community. This book is a must-read in today's surveillance-filled world: buy it for your friends, get it from the library. Spread the word.

Colorless Tsukuru Tazaki and His Years of Pilgrimage

Apurva Shukla's RSS Feed — Sun, 12 Jul 2015 18:30:39 GMT

⭐ ⭐ ⭐ ⭐ ⭐ Eerily sinister, yet exquisitely beautiful. One cannot explain via words the feelings Murakami struck with this remarkable story.…

Inside Out

🦉 olshansky 🦁 — Sun, 12 Jul 2015 11:42:40 GMT

Olshansky's review of Inside Out

The Good Dinosaur

🦉 olshansky 🦁 — Sun, 12 Jul 2015 10:28:24 GMT

Olshansky's review of The Good Dinosaur

The Alchemist

🦉 olshansky 🦁 — Sun, 12 Jul 2015 03:00:00 GMT

Olshansky's review of The Alchemist by Paulo Coelho

Video - Cluster Failover Performance Demo

smcleod.net — Sun, 12 Jul 2015 03:00:00 GMT

Hadoop on Ceph: diving in

Noah Watkins — Sun, 12 Jul 2015 03:00:00 GMT

Using Hadoop to process data stored in the Ceph distributed file system.

Video game transcendental meditation

Jason Eckert's Website and Blog — Sun, 12 Jul 2015 03:00:00 GMT

What is transcendental meditation?

Transcendental meditation is the most commonly practiced form of meditation in the world. People who practice it often listen to, or repeat, a particular sound or mantra in order to reach a relaxed state of consciousness. It’s been praised by many for decades as a means of lowering stress and anxiety.

What is a “flow state” within a video game?

Video games that require quick thinking, fast reflexes and precision are often difficult to excel at - but with practice, the game can be mastered. When mastered, the player’s physical precision is almost effortless (encoded in “muscle memory”), the the thought processes seem perfectly natural. It’s as if your mind is actually “in” the game. This state is often called “the zone” by gamers, or a “flow state” by psychologists.

systemd, User Instances, Device Units, and Media Archiving

Tyblog — Sat, 11 Jul 2015 09:00:00 GMT

I recently used systemd, HandBrake, and some simple scripts to digitize a large collection of physical media (for personal, archival use.) In this post I'll go through systemd features that made this easier and cover all the components that make the automated pipeline work.

If systemd, automation, or digital archiving sounds interesting to you, then read on!

The Problem
Hardware
systemd
The Script
Conclusion

The Problem

Atoms take up space. I'm moving in the near future, so my wife and I have been simplifying and trying to convert whatever we can into digital media rather than physical. This includes a moderately-sized DVD collection - which would be really nice to convert into media files so they can be stored on my NAS and used in tandem with my already-existing Kodi setup.

However, there's a bit of a vacuum when it comes to automating this sort of process. Handbrake makes this easier, but the extent of the interaction I want a human to have to perform is to 1) put a DVD in and 2) take the DVD out - no selecting a title track, choosing an encoding format, and so on.

We can make computers do the tedious parts!

We'll start from the ground up, beginning with device activation and ending with media file processing.

Hardware

I have a NAS running Arch Linux which serves as a multi-purpose always-on server. I also had a spare DVD drive laying around after tearing it out of a 2009 Macbook Pro to add a second hard drive, so the first step was to convert that formerly internal drive into a USB-connected external drive.

This is pretty easy by putting the drive into an external enclosure, which I can hook into the NAS as a USB device.

systemd

Devices

Because Arch Linux uses systemd through-and-through, the init system also manages devices. You can actually see managed devices with the following command on a systemd-managed distribution with:

shell

systemctl --all --full -t device

This lists all 'active' devices, which in systemd nomenclature indicates the device is up, running, and available. In our case, once a CD/DVD is loaded, the dev-cdrom.device device is activated and started to make it available.

We need a way to trigger the activation of a script, and this seems like a good system to hook into. Note that this is also possible with udev rules - the hotplugging mechanism that permits you to write rules that can dictate how devices are mounted and made available to a Linux operating system. However, if we look at the man page for udev, we find this:

RUN{type}
    Add a program to the list of programs to be executed after processing all the rules for a specific event ...

    ...snip...

    This can only be used for very short-running foreground tasks. Running an event process for a long period of time may block all further events for this or a dependent device.

Definitely not a good candidate for a script that can potentially run for a few hours to properly encode optical media.

However, systemd can fit this need pretty well. The feature set of systemd exceeds the scope of this post, but there's good documentation under the following man pages: systemd.exec, systemd.service, and systemd.unit.

The Service

We'll define our script as a systemd service, and work under the paradigm that starting our service begins capturing our DVD.

Here's the content of the service unit:

Systemd

Font used to highlight builtins.
Font used to highlight keywords.
Font used to highlight type and class names.

[Unit]
Description=Automatically rip inserted DVDs
After=dev-cdrom.device
BindsTo=dev-cdrom.device
Requisite=dev-cdrom.device

[Service]
WorkingDirectory=/path/to/where/you/want/media/files
Type=oneshot
ExecStart=/usr/local/bin/rip-dvd.sh
ExecStart=/usr/bin/eject /dev/cdrom
StandardOutput=journal

[Install]
WantedBy=dev-cdrom.device

Let's go over the parts that aren't self-explanatory – we'll start with WantedBy, as that's the most important part for our task. Note that each of these options are well-explained in the man pages as well:

WantedBy - This indicates that the dev-cdrom.device unit will ask for our service to be started when the device itself gets activated. We say WantedBy and not RequiredBy because the latter would indicate that our service is a required dependency of making our cdrom available - which isn't the case (there's no logical relationship that indicates our autorip service is doing something that must be available for the cdrom to work)
After, BindsTo, Requisite - These seem somewhat redundant, but let's look at each:
- Requisite - This indicates that the cdrom needs to be running in order for our unit to function. It's similar to using "Requires", except that if our cdrom isn't running, systemd won't try to "start" is as part of beginning our ripping process. This makes sense, as if the disk isn't in the optical drive, systemd can't do much physically to insert the disk and make the cdrom device unit active.
- After - Ensure that our service begins only after the cdrom unit has started completely. This is important, because otherwise systemd will attempt to aggressively parallelize unit activation once this unit gets triggered by our WantedBy line.
- BindsTo - This isn't a strictly required option, but it makes the implementation cleaner. Essentially our unit will be killed if the indicated unit disappears from some reason - for example, if the disk gets ejected from the optical drive (which makes sense - without a disk to read from, our ripping process should be halted.)
Type=oneshot - This unit is intended to run then exit without any persistent daemon. This line tells systemd to expect that behavior (our script will encode the DVD, then exit.)
StandardOutput=journal - In my testing I noticed that I didn't always get script output captured in the journal. Explicitly adding this ensures that any script output is automatically logged to the journal, and can then be easily retrieved with journalctl --user-unit <unit> (if you've tried to debug cron job behavior, you'll recognize this as pretty useful.)
ExecStart - For a oneshot service, we can define multiple stanzas. In this case, we wrap up the ripping process into a script, then if that command succeeds, spit the disk out. By default multiple ExecStart lines are dependent upon the success of the preceding execution, so the unit will halt and not eject the disk if ripping fails.

The most important line of configuration here is WantedBy. Because systemd has abstracted both devices and services into unit files, we can create arbitrary relationships such as this one. Asking a unit to begin as the result of a device starting then becomes a trivial task, as opposed to using udev's RUN, which could cause problems by inlining scripts into the pipeline that actually initializes the device.

Enabling the Service

We now have a systemd service file defining the script we want to run as a result of inserting a disk. How do we enable it?

One option is to install it system-wide into /etc/systemd/system, but we're going to use a user instance for this script. Partially to get familiar with user instances, and partially because it fits our use case: I, as a user, want to run this script based on a trigger. It also follows the principle of least privilege and I have all my dotfiles tracked anyway, so this process becomes nicely version controlled.

Luckily this is pretty straightforward. By placing the unit in /home/$user/.config/systemd/user/autorip.service (I've named my unit "autorip" here), the user systemd instance becomes aware of the service unit.

Note: This was done on an Arch Linux machine which is thoroughly steeped in systemd, so I'm not sure what the steps may be to leverage systemd user instances on another distro. I will mention that after some recent upgrades, I had to follow the Arch Wiki instructions to enable user instances to persist after log-out, otherwise enabled services disappear once the user session is destroyed.

Now that our user instance is aware of the unit's presence, just enable it:

shell

systemctl --user enable autorip.service

This means that, among other things, our WantedBy setting takes effect, asking to be activated when the cdrom unit is activated.

That's it! Now when the cdrom is inserted, our unit will get triggered, and we get useful script output logging for free.

The Script

Let's talk about the script we need. I initially turned to HandBrake, which is an excellent tool for media ripping that also has a command-line utility. I tried a few scripts with it, which worked, but were difficult to get working with a ratio of size to quality that I wanted.

It turns out Don Melton has some absolutely fantastic scripts that leverage Handbrake for very well-tuned video encoding. I homeshick clone-ed this and symlinked the scripts into my homeshick-tracked ~/.bin $PATH directory, so my user can call on transcode-video.sh properly.

All that remains is to glue some pieces together to actually encode the video. Here's a simplified version of the script I wrote. Some parts to note:

I'm using zsh - some features like that ${(C)..} syntax can be shell-specific. (that capitalizes a string, by the way)
filmdate.py is a small utility I wrote to attempt a guess at the year of a film's title. The source is on github if you're interested. I used this because there's little (if any) metadata available on the disk to rely on, and most post-processors need, at minimum, the title and year to parse what movie the filename contains.
The WorkingDirectory setting in the unit is a temporary directory that I can write the file in, and the last mv in the script drops the finished file into a directory that my setup watches to pick up media.

If you've ever tried automation like this, you know that the amount of information you can glean from a plain DVD is pretty bare; essentially only the (sometimes mangled) title is available. In my script I attempt to pull the title from Handbrake, which can sometimes fail. In that case you'll notice I use := variable assignment. This allows me to use the set-environment action in systemd before inserting a disk if I want to statically set the film title before attempting to rip a disk. Thus if I know that a particularly ugly title is going into the DVD drive, I can use:

shell

systemctl --user set-environment TITLE="Justin.Bieber.Never.Say.Never"

This means that my script will not overwrite the TITLE variable if it finds one in the environment. This is a little clunky because we set set this globally across all user units instead of for our specific unit, but it works well enough. Issuing

shell

systemctl --user unset-environment TITLE

Means the script will fallback to using Handbrake to attempt auto-detection. SELECT_TITLE behaves similarly if the automation can't determine the correct title.

Conclusion

With the script in the right location for our unit's ExecStart and the .service enabled, DVDs inserted into the system will trigger activation of our user service. In my case, the first ExecStart should succeed, moving the file into a pickup location for Kodi, and then eject the disk. This means that digitization is just a matter of inserting the disk and taking it out when I get a Pushover notification from Couchpotato that my media has been captured correctly.

There's lots of room for improvement here - for example, parsing DVD titles can be finicky, which still requires manual intervention to fix by setting environment variables for the unit. However, the process is much more streamlined than using a point-and-click method, and helped me archive a pretty big stack of media with very little human interaction.

A full play through of my HTML5 game, CanyonRunner

Zachary Proser — Sat, 11 Jul 2015 03:00:00 GMT

I built a complete game using the Phaser.js game development framework

Bosnia and Hercegovina travel bookmarks

xenodium.com @alvaro — Sat, 11 Jul 2015 03:00:00 GMT

Ten reasons to visit Bosnia & Hercegovina.

Ireland travel bookmarks

xenodium.com @alvaro — Sat, 11 Jul 2015 03:00:00 GMT

Skellig Michael.
Fishy Fishy in Kinsale: beautiful town on the water.
Belfast.
Giant's Causeway.
Greyhound dog races at Shelbourne Park.
Old library chamber, Trinity College, Dublin.

AWS EC2 Instance Types to Use as Test Agents

Ben E. C. Boyter — Fri, 10 Jul 2015 11:03:10 GMT

When you are running test agents on AWS knowing what instance type to run as test agents (for TeamCity or otherwise) can involve a lot of trial and error. Not only can there be great savings to be made by picking the correct instance type you can speed up your builds and get test feedback back faster which can be far more valuable the cost of a few additional cents an hour.

Interview: Die E-Commerce-Plattform von GALERIA Kaufhof

Home on The Log Book of Manuel Kießling — Thu, 09 Jul 2015 18:13:00 GMT

In Ausgabe #23 der ShopTechTalks unterhält sich Roman Zenner mit Michael Wegener (talentformation GmbH) und mir über die selbst entwickelte E-Commerce-Plattform von GALERIA Kaufhof. Wir berichten unter anderem über die nötigen Schritte von einer monolithischen Shop-Lösung, die primär von einem externen Dienstleister weiterentwickelt und gepflegt wurde hin zu einem verteilten System, an dem interne Teams arbeiten:

Pizza in London

xenodium.com @alvaro — Thu, 09 Jul 2015 03:00:00 GMT

Not tried these yet. Taking note:

Bravi Ragazzi (Streatham).
Homeslice (Covent Garden).
Lord Morpeth (Hackney).
Santa Maria (Ealing).
Voodoo Ray's (Dalston).
Well Kneaded Wagon (Date-dependent location).

mp4 to gif

xenodium.com @alvaro — Thu, 09 Jul 2015 03:00:00 GMT

Converting mp4 to gif is handy for posting short screencasts. You can convert to gif using ffmpeg and optimize with imagemagick. To install:

apt-get install ffmpeg imagemagick (linux)
brew install ffmpeg imagemagick (Mac)

Convert to gif:

ffmpeg -i my.mp4 -pix_fmt rgb24 -r 5 my.gif

Optimize with:

convert -dither none -layers Optimize my.gif my_optimized.gif

UPDATE: There's also licecap and subsequently optimize with:

cat source.gif | gifsicle --colors 256 --optimize=3 --delay=15 > target.gif

UPDATE: Also consider for .mov:

ffmpeg -i in.mov -pix_fmt rgb24 -r 10 -f gif - | gifsicle --optimize=3 --delay=3 > out.gif

Fantasy News

Sam Solomon — Thu, 09 Jul 2015 03:00:00 GMT

Fantasy News is a year-round community for fantasy sports fans.

Exploring Microservices Architecture on Heroku

Brightball Articles — Thu, 09 Jul 2015 02:17:02 GMT

Building an application with a microservice architecture is an excellent long-term decision if you can afford the increase in upfront time investment to do it properly. Heroku provides a platform that most developers know for simple deployment, but it also dramatically simplifies microservices architecture.

The Drawing of the Three (The Dark Tower, #2)

Apurva Shukla's RSS Feed — Wed, 08 Jul 2015 11:18:05 GMT

⭐ ⭐ ⭐ ⭐ ⭐ I loved every single second of the book! While this book serves to continue the storyline, it does so wonderfully.

Issues with Google's Bug Prediction Algorithm

Ben E. C. Boyter — Wed, 08 Jul 2015 10:58:57 GMT

December 2011 the Google Engineering team published a blog post about bug prediction at Google. The topic caused quite a lot of discussion at the time over the internet on forums such as Hacker News and the Reddit programming sub-reddit.

How bug prediction works

In a nutshell the prediction works by ranking files against checking the file commit history and seeing how many changes have been flagged as bug fixes. Of course this means that code which was previously buggy will still appear in the list. This issue was also addressed in the post and the results were weighted over time to deal with this issue.

Oh-My-Vagrant “Mainstream” mode and COPR RPM's

The Technical Blog of James on purpleidea.com — Wed, 08 Jul 2015 04:14:19 GMT

Making Oh-My-Vagrant (OMV) more developer accessible and easy to install (from a distribution package like RPM) has always been a goal, but was previously never a priority. This is all sorted out now. In this article, I’ll explain how “mainstream” mode works, and how the RPM work was done. (I promise this will be somewhat interesting!)

Prerequisites:

If you haven’t read any of the previous articles about Oh-My-Vagrant, I’d recommend you start there. Many of the articles include screencasts, and combined with the examples/ folder, this is probably the best way to learn OMV, because the documentation could use some love.

CentOS 7 and HA

smcleod.net — Tue, 07 Jul 2015 03:00:00 GMT

First some background…

One of the many lessons I’ve learnt from my Linux HA / Storage clustering project is that the Debian HA ecosystem is essentially broken, We reached the point where packages were too old, too buggy or in Debian 8’s case - outright missing.

In the past I was very disappointed with RHEL/CentOS 5 / 6 and (until now) have been quite satisfied with Debian as a stable server distribution with historicity more modern packages and kernels.

Adventures after Google I/O-3(GCM).

Prasham H Trivedi — Tue, 07 Jul 2015 03:00:00 GMT

After trying and playing some new stuffs(except M preview) released in Google I/O, I have found some trivial things to take care of. In this part, we will see how GCM 3.0 gives advantage to server team to send notifications and how implementing it affects Android Development.

WCF net.tcp test client in PowerShell

Allan's Blog — Mon, 06 Jul 2015 13:39:39 GMT

Here's a convenient way of creating a WCF net.tcp client for a quick test. It uses PowerShell to compile C# code to an exe on the fly. Note that the interface can be partial (only containing the method you want to call).

$source = '
    using System;
    using System.ServiceModel;

    [ServiceContract(Namespace = "http://example.com/SomeSchema")]
    public interface ISomeService
    {
        [OperationContract]
        bool SomeMethod();
    }

    public class Wcftest
    {
        public static void Main(string[] args)
        {
            var binding = new NetTcpBinding();
            var address = new EndpointAddress("net.tcp://" + args[0] + ":808/SomeService");

            var factory = new ChannelFactory<ISomeService>(binding, address);
            var proxy = factory.CreateChannel();

            Console.WriteLine(proxy.SomeMethod());
        }
    }
'
Add-Type `
    -ReferencedAssemblies 'System.ServiceModel.dll' `
    -TypeDefinition $source `
    -Language CSharp `
    -OutputAssembly 'c:\windows\temp\wcftest.exe' `
    -OutputType 'ConsoleApplication'

c:\windows\temp\wcftest.exe "example.com"
del c:\windows\temp\wcftest.*

What is Usability Testing?

Ben E. C. Boyter — Mon, 06 Jul 2015 10:57:30 GMT

Usability tests are manual tests used to check that the user interface is understandable. The focus of the tests are to ensure that product meets its intended purpose. These sort of tests can be subjective and are usually impossible to automate. It is important to differentiate usability testing from simply showing an interface to someone and asking them “Do you understand how this?”. It is usually done by creating a scenario such as “Can you find and add this song to a new playlist” and observing the steps that the user takes to perform the task.

Keyboards bookmarks

xenodium.com @alvaro — Mon, 06 Jul 2015 03:00:00 GMT

Leather Backpack

june.kim — Mon, 06 Jul 2015 03:00:00 GMT

The One Question Every Prospective College Student Should Be Asking

Tilting at Windmills — Mon, 06 Jul 2015 03:00:00 GMT

Apologies for the clickbaity headline. That sort of thing isn't usually my style. I'll cut right to the chase. The question is "What…

July 2015 Project List

Venam's Blog — Patrick Louis (Lebanon) — Mon, 06 Jul 2015 00:00:00 GMT

Hello fellow readers, In this post I'll list some of the projects I'm currently working on.

Parallels

🦉 olshansky 🦁 — Sun, 05 Jul 2015 17:13:24 GMT

Olshansky's review of Parallels

Mutation Testing or How to Test Tests

Ben E. C. Boyter — Sun, 05 Jul 2015 10:56:03 GMT

Mutation testing is a technique used to verify that tests are providing value. Mutation testing involves modifying the given program in small ways. These could include changing boolean checks such as if a condition is True to being False. A mutated version of code is known as a mutant. For each mutant a the test suite is run against it. The tests when run over the mutant version should have a percentage of failure. Where a mutant is not caught additional tests can be written to cover these cases.

United States travel bookmarks

xenodium.com @alvaro — Sun, 05 Jul 2015 03:00:00 GMT

Lebanon travel bookmarks

xenodium.com @alvaro — Sun, 05 Jul 2015 03:00:00 GMT

Slovenia travel bookmarks

xenodium.com @alvaro — Sun, 05 Jul 2015 03:00:00 GMT

Lake Bohinj.

Belgium travel bookmarks

xenodium.com @alvaro — Sun, 05 Jul 2015 03:00:00 GMT

Leather Valet Tray

june.kim — Sun, 05 Jul 2015 03:00:00 GMT

Travis CI and Sauce Labs

kyrofa's blog — Sun, 05 Jul 2015 03:00:00 GMT

I think we can all agree that testing one’s Rails project is a good thing. I think we can also agree that those tests should be run on a regular basis (i.e. after every change). I think it’s a logical conclusion then that having a continuous integration environment setup for one’s project would be considered ideal. This is really pretty easy for open-source projects, since a number of companies provide free support for them.

Who is Responsible for Software Quality?

Ben E. C. Boyter — Sat, 04 Jul 2015 10:54:08 GMT

In the beginning of my software development career I was interviewing for an intern position at Microsoft. I never did get the job but one think out of that interview process really stuck with. The second interviewer after the usual getting to know you chat aded me the following question. “On any given software project we have developers, software testers / quality assurance and managers involved. Who is responsible for the quality of the software?”. Being young and naive I confidently responded that the QA/testers were. After a long discussion artfully controlled by the interviewer I came to change my opinion. Below is the line of reasoning I went through with him.

Talks

Ben E. C. Boyter — Sat, 04 Jul 2015 09:12:08 GMT

A few of the talks I have given and the slide desks are included below.

Python @ searchcode.com sypy_searchcode.key (keynote) sypy_searchcode (powerpoint)
Decoding CAPTCHA’s for Fun and Profit DecodingCaptchas Reveal.js (futher details can be found http://www.boyter.org/2015/03/decoding-captchas-presentation/)
How AWS and Go Transformed a Public Broadcaster https://boyter.org/static/aws-go-archive-presso/
Sloc Cloc and Code - Can a crusty Go program outperform a well written Rust Project? https://boyter.org/static/gophercon-syd-presentation/ https://www.youtube.com/watch?v=jd-sjoy3GZo
DataEngBytes 2023 - Processing 40 TB of code from ~10 million projects with a dedicated server and Go for $100 https://boyter.org/static/dataenbytes2023/
GopherCon 2023 - Bloom Filters: Building a Cutting Edge Go Search Engine to Explore the World’s Source Code https://boyter.org/static/gophercon-syd-presentation-2023/

The Essays of Warren Buffett : Lessons for Corporate America

🦉 olshansky 🦁 — Sat, 04 Jul 2015 03:00:00 GMT

Olshansky's review of The Essays of Warren Buffett : Lessons for Corporate America by Warren Buffett

Starting Elm

λm.me — Sat, 04 Jul 2015 03:00:00 GMT

Edit: this is old and probably no longer accurate!

** This post has been revamped and added to the Elm website, you can read it here. **

In this post I will try to guide you through some first steps of writing Elm.

First, we'll talk about the installation process and the tools that come with it, but since Elm has a pretty good Online Editor you can definitely skip it for now.

After that we'll talk about actually learning how to write Elm programs. The Elm website has a lot of information in it and it's not always clear where to start, so this guide will try to put these heap of information into order.

Installation

First, visit the Install page on the elm site, download an installer for your platform and install Elm-platform, which will install the following tools:

elm
elm-package
elm-make
elm-repl
elm-reactor

Let's go over them one by one:

elm

elm is actually a way to run all other tools. Try opening a terminal and run elm to see the help message.

elm-package

elm-package is a package managing tool for elm, making it easy to install and publish packages to and from package.elm-lang.org.

When starting a new elm project, run:

elm package install

This will install the elm-core package and will create an elm project file: elm-package.json.

In elm-package.json we state information on the project, such as: the project name, author, license, dependencies, etc.

Interesting commands:

install: install dependencies in elm-package.json
publish: publish your library to package.elm-lang.org
bump: bump version numbers based on API changes
diff: get the difference between two APIs

elm-make

elm-make is used to compile elm code (file or project) into HTML or JavaScript.

When compiling a file (for example: Main.elm) into an HTML file (for example: index.html), you probably want to write something like this:

elm make Main.elm --output=index.html

Interesting flags:

--warn: Prints warnings to improve code quality

elm-repl

elm-repl is a Read-Evaluate-Print Loop tool for Elm. In it you can write elm expressions and evaluate them using Node.js. You can also import code and libraries, but for obvious reasons browser related stuff will not work.

Important commands:

:help: will print a help message
:quit: quit the repl

elm-reactor

elm-reactor is an interactive development tool for elm. With elm-reactor you can run elm code without using elm-make to compile them first, hot swapping and time travel debugging.

Running elm reactor will open a web server on 0.0.0.0:8000 where you can go to from any browser and select the file you want to run. If you want to use elm reactor's abilities, press the wrench next to the name of the file on the left. It will open the file with a column on the right where you can use those capabilities.

Important flags:

-a=ADDRESS: Change the address in which elm reactor runs. Since the default address, 0.0.0.0 is not supported on browser such as chrome and is broadcasting to anyone, I recommend always using -a=localhost
-p=PORT: Change the port in which elm reactor runs.

So for example, if you want to run the elm reactor, write:

elm reactor -a=localhost

Learning Elm

In the documentation page on the website, you can find a bunch of resources on elm.

For beginners, I recommend first starting with the Elm Complete Guide (link to part 1) and play with what you learn on the elm-repl or Online Editor. Writing Elm is essential to understanding it, and Elm Provides you the tools to easily do so.

After the Elm Complete Guide, you might want to read about Elm's Syntax and Style Guide. (an important thing to note is that there are constructs in Elm that are indentation sensitive.)

Checkout the many examples in the Examples page.

Read the Elm Architecture Tutorial.

You can also go over the cs223 Functional Programming course by the University of Chicago which has many good tutorials on elm and purely functional data structures, but bear in mind that the course uses Elm 0.14.1, which might be a little different. No worries! That's why you read the complete guide and the syntax guide.

Read more tutorials like Elm for the Frontend, Right Now, Checkboard Grid Tutorial and Building HTML by Parsing Parameters and others you can find on /r/elm and the mailing list.

But most importantly, don't forget to write code! Here are a few ideas for simple projects for you to get started:

An elm program that writes Right or Left in the middle of the screen depending on whether the mouse cursor is on the left half of the screen or right half of the screen
An elm program that displays dots randomly on the screen, with reset and pause/play buttons
An elm program that displays the list of repositories of a github user entered on a text field
Snake clone (extra: add highscore)

And, if you ever get stuck, try posting on the mailing list or come to #elm IRC channel on irc.freenode.net

Heroku PostgreSQL vs Amazon RDS for PostgreSQL

Brightball Articles — Fri, 03 Jul 2015 14:04:23 GMT

PostgreSQL is becoming the relational database of choice for web development for a whole host of good reasons. That means that development teams have to make a decision on whether to host their own or use a database as a service provider. The two biggest players in the world of PostgreSQL are Heroku PostgreSQL and Amazon RDS for PostgreSQL. Here's a detailed comparison.

A story about Hubris and Integration Tests

Ben E. C. Boyter — Fri, 03 Jul 2015 10:51:07 GMT

Philip Dormer Stanhope, 4th Earl of Chesterfield (pictured) managed to embarrass me in front of my peers once. Sort of. In truth it was my hubris that caused the incident. Here is how it happened and what I learnt through the process.

In the summer of 2010 I was tasked with developing a new application where I worked. The requirement was fairly simple “We need a web application to upload a CSV”. Requirements such as this aren’t exactly conducive to a good outcome but I was confident that given the data required to upload it would be fairly easy to do.

Fishing with Emacs

xenodium.com @alvaro — Fri, 03 Jul 2015 03:00:00 GMT

OK not quite, but having recently learned about C-M-w (append-next-kill), I used it in a keyboard macro to fish out matching lines. This is similar to flush-lines, except the kill ring is also populated. This is handy, if you need the flushed lines. Here's an example.

Here's the equivalent in Emacs lisp:

(defun flush-kill-lines (regex)
  "Flush lines matching REGEX and append to kill ring.  Restrict to \
region if active."
  (interactive "sFlush kill regex: ")
  (save-excursion
    (save-restriction
      (when (use-region-p)
        (narrow-to-region (point) (mark))
        (goto-char 0))
      (while (search-forward-regexp regex nil t)
        (move-beginning-of-line nil)
        (kill-whole-line)))))

California travel bookmarks

xenodium.com @alvaro — Fri, 03 Jul 2015 03:00:00 GMT

General Sherman Tree at Sequoia National Park.

Shiny Apps

Playground — Thu, 02 Jul 2015 03:00:00 GMT

This a collection of small shiny apps I have made (or going to make) to learn shiny (by RStudio). The list of apps in this repo are listed (and documented) on this page. To learn more about the apps (what they do and how were they developed to do what they do) just read further.

Update on Rigel and response from DjurAkuten

Liza Shulyayeva — Wed, 01 Jul 2015 14:41:58 GMT

Context: The Month from Hell.

DjurAkuten

After numerous attempts David finally got through to DjurAkuten, the clinic that did Rigel’s neutering surgery and decided to put a catheter in him because they thought his bladder looked big - the one he left with a urinary infection and an inability to pee.

How searchcode.com is Unit and Integration Tested

Ben E. C. Boyter — Wed, 01 Jul 2015 10:46:18 GMT

searchcode is a source code and documentation search engine. It allows users to is search over 20 billions lines of code and millions of API documentation. It at the time of writing gets over 300,000 unique visitors a month processes many millions of API requests.

What follows is how I am testing searchcode, the issues I hit and where I think I am getting the most value. As big believer in testing as a methodology of improving code quality, one of my goals when rewriting searchcode was to ensure that it had a comprehensive test suite.

Rebind caps lock to control key on Mac OS X

xenodium.com @alvaro — Wed, 01 Jul 2015 03:00:00 GMT

Let's see if this one sticks. I'll give caps lock as control a try. Rebinding the keys on Mac OS X is easy enough:

System Preferences -> Keyboard -> Keyboard Tab -> Modifier Keys…

Flaky Tests

Ben E. C. Boyter — Wed, 01 Jul 2015 01:50:04 GMT

A test is considered flaky or flakey if it if fails occasionally. Generally flaky tests are considered to be a bad thing and should be modified to ensure they work correctly every time. This is because a test that is not trustworthy will be ignored even when indicating real failure.

There are many situations that can cause to become flaky. Integration and acceptance tests are generally the tests in your test suite most likely to become flaky. They generally have more integrations across your software stack and as such there are more things likely to go wrong. We going to go through a few of the main reasons and go through in detail what you can do about a specific one.

Clean Testable Repository Data Access in C Sharp

Ben E. C. Boyter — Wed, 01 Jul 2015 01:44:13 GMT

Below is an implementation of an extremely clean data access pattern possible using C# and Entity Framework. It saves you the effort of mocking the database context as the code you end up writing is so simple it is all compile time checked.

The advantages of this are firstly that everything is very easy to test as you can perform all joins in your service layer with mocks of the repository. Secondly it makes your data layer stupidly simple allowing you to forgo writing many tests which would provide little value.

Passing Static Credentials Upstream through NGinx: Making NGinx add credentials to upstream requests

Joshua Rogers — Tue, 30 Jun 2015 15:00:00 GMT

Just over a year ago we went over how to setup a reverse proxy that would require our users to authenticate. In our original proxy article, we generated some NGinx configuration that looked a bit like… location / { proxy_pass http://jira; proxy_set_header Host $host; proxy_set_header X-Real-IP $remote_addr; proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for; } And it worked. And everyone was happy. Until the day the request came in that NGinx should add a fixed set of credentials to all upstream web requests.

Searchable ebooks in Emacs

xenodium.com @alvaro — Tue, 30 Jun 2015 03:00:00 GMT

If you haven't bought Mastering Emacs by Mickey Petersen, you should. It's a wonderful source of Emacs tips. Having just finished the ebook on my Kindle, I was keen to go back and fish out some of that newly found wisdom. My immediate reaction was to figure out a way to make the ebook searchable from Emacs.

The ebook is available in epub and pdf format. Though Emacs's docview is super handy for viewing pdf's, searching didn't feel as comfortable as searching in org mode. The epub, on the other hand, proved useful. Pandoc can easily convert from epub to org.

pandoc  --from=epub --to=org mastering-emacs.epub > mastering-emacs.org

After a some tidying (mostly removing BEGIN_HTML/END_HTML blocks and adding TITLE/AUTHOR), the resulting org file is surprisingly clean and easy to search/navigate. helm-swoop and helm-org-in-buffer-headings are great for that.

Cerberus 0.9 has been released

Nicola Iarocci — Tue, 30 Jun 2015 03:00:00 GMT

A few days ago Cerberus 0.9 was released. It includes a bunch of new cool features, let’s browse through some of them.

Collection rules

First up is the new set of anyof, allof, noneof and oneof validation rules. anyof allows you to list multiple sets of rules to validate against. The field will be considered valid if it validates against one set in the list. For example, to verify that a property is a number between 0 and 10 or 100 and 110, you could do the following:

The long road to Async/Await in JavaScript

Thomas Hunter II — Tue, 30 Jun 2015 03:00:00 GMT

The Hybrid World is Coming

Tanel Poder Blog — Tue, 30 Jun 2015 02:14:13 GMT

Here’s the video of E4 keynote we delivered together with Kerry Osborne a few weeks ago.

It explains what we see is coming, at a high level, from long time Oracle database professionals’ viewpoint and using database terminology (as the E4 audience is all Oracle users like us).

However, this change is not really about Oracle database world, it’s about a much wider shift in enterprise computing: modern Hadoop data lakes and clouds are here to stay. They are already taking over many workloads traditionally executed on in-house RDBMS systems on SAN storage arrays – especially all kinds of reporting and analytics. Oracle is just one of the many vendors affected by all this and they’ve also jumped onto the Hadoop bandwagon.

The Unsung Benefits of Software Testing

Ben E. C. Boyter — Tue, 30 Jun 2015 01:35:06 GMT

One benefit that is generally not talked about when discussing testing is the following. The feeling of productivity because you are writing lots of code.

Think about that for a moment. Ask any developer who wants to develop why they became a developer. One of the first things that comes up is “I enjoy writing code”. This is one of the things that I personally enjoy doing. Writing code, any code especially when its solving my current problem makes me feel productive. It makes me feel like I’m getting somewhere. Its empowering.

Snaily updates

Liza Shulyayeva — Mon, 29 Jun 2015 21:32:35 GMT

I was going to try to remember all the things I’ve done in the last month on snails, but this seems impossible. So here’s a lazy list of git commits within this time:

Persistent undo functionality in vim

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Mon, 29 Jun 2015 21:02:55 GMT

Sometimes, it is the small things that make my day. I recently discovered that recent versions of vim offer a persistent undo functionality. This means I can edit my files, close them, and still be able to undo changes when I open them the next time. How cool is that?

The activation could not be simpler. Just place the following in your .vimrc:

set undofile
set undodir=$HOME/.vim/undo

The file path in undodir should refer to an actual directory in your .vim directory, so it might need to be created first. Furthermore, you can use the variables undolevels and undoreload to specify how many changes can be undone at most and how many modified lines should be saved, respectively.

I love vim more and more every day.

The Virtuous Traitor

Pete's Points — Mon, 29 Jun 2015 05:38:19 GMT

I was just reading some articles and comment threads debating the issue of git merge verses git rebase. I think it's a good case study for a decision-making methodology of actually trying both sides. I think outside of software development, the cost of trying both sides of a debate can be high in terms of time, effort, and money. Let's say hypothetically there are several competing theories of long-term crop management for farmers. It would take many years, lots of technical know-how, and lots of dedication to split a large farm into portions, each serving as a guinea pig for a particular crop management technique. However, in software development, many of these techniques can easily be tried on some small/fast basis like a side project, a particular feature, etc.

For example, my web site has a mix of content and features that make it debatable as to whether a real dynamic application is warranted or would a static site generator approach suffice. I bypassed a lot of debate by just running it both ways for a meaningful amount of time, and at the end it was clear to me which way I preferred.

When you see some of these debates, for example:

git vs mercurial
merge vs rebase
functional vs OO
shell scripts vs configuration mgmt
use DB constraints heavily or lightly
language X vs language Y
tool X vs tool Y
Would I prefer consulting to full-time employment?

It's good to think about how much time and effort would it take to actually give each option an honest, non-trivial try. Often I find it's worth it and after the fact I have real first-hand knowledge to show for it. I'm particularly fond of first-hard knowledge/experience and skeptical of opinions held based on reading/hearing third-party opinions.

Here's a few recent things I put through this methodology:

Is coffeescript worth using?
Are docker containers interesting and useful?
Is ansible better than my existing shell scripts?
Would Atom be any better than Sublime?

Based on the cases made in the articles I was reading today, I plan to try a rebase-driven methodology on my next git project given I've mostly been doing merges and see how it goes.

Play for both teams. It's often the best way to guide a decision. And don't forget that after a while things change and sometimes we need to re-visit old preferences.

Testing In Software Engineering

Ben E. C. Boyter — Mon, 29 Jun 2015 01:38:23 GMT

Testing is software engineering is a method of providing information about code quality when developing a piece of software. The intent of writing and running tests is the enforce good software design and identify software bugs and defects. These defects can include specification/requirement errors as well as developer mistakes.

The general aim of software testing is the ensure that software meets the following goals,

Meets its requirements gathered before its design and implementation

Grouping Tests: Unit/Integration vs Fast/Slow Tests

Ben E. C. Boyter — Mon, 29 Jun 2015 01:36:47 GMT

There is a great deal of argument in the testing community over how to label tests. One camp likes to label tests using levels such that unit tests are in one group, integration in another and so forth. The other likes to label them based on how long they take to run ignoring what level they are in. Fast tests are those that run in milliseconds while slow take longer then this. The reason this is important is that when adopting a testing process slow and flaky tests (those which fail often) are the enemy. Slow tests tend to be run less often in the development process. A delay of a few seconds can seriously interrupt a developer or testers workflow. Also the more often your tests fail randomly the less confidence you are likely to have in them, ignoring genuine errors until they fail multiple times.

Sanity Testing

Ben E. C. Boyter — Mon, 29 Jun 2015 01:13:41 GMT

Software sanity tests are closely associated with smoke tests. They attempt to determine if is reasonable to continue with testing a given piece of software. The objective is not to test all functionality, but to determine if there is value in doing so. You can consider it a “Should I continue to test this?” check. Sanity tests differ from smoke tests as they exist to check if new functionality has been met and existing bugs have been resolved.

Silicon Valley: Season 1

🦉 olshansky 🦁 — Sun, 28 Jun 2015 07:57:16 GMT

Olshansky's review of Silicon Valley: Season 1

Jurassic Park

🦉 olshansky 🦁 — Sun, 28 Jun 2015 07:48:51 GMT

Olshansky's review of Jurassic Park

Generic Programming: The Concept

Whacky Labs — Sat, 27 Jun 2015 20:58:54 GMT

Problem

Say, we have a type Animal which is at this time just an idea, so a protocol.

protocol Animal {
    func say()
}

Next, say we want a generic function that works with Animal.

func greet<T: Animal>(animal: T) {
    animal.say()
}

Looks good. Now let’s create our concrete types.

class Mammal: Animal {
    func say() {
        println("Hello! I'm a mammal!")
    }
}

class Reptile: Animal {
    func say() {
        println("Hello! I'm a replite")
    }
}

And bring it to use

let m = Mammal()
let r = Reptile()
let animals: [Animal] = [m, r]

for animal in animals {
    greet(animal)

}

Oh o! Unfortunately this code won’t compile. This is the error the compiler might throw on your machine.

error: generic parameter 'T' cannot be bound to non-@objc protocol type 'Animal'

Let’s rewrite our implementation in C++ and see what sort of error we get.

class Animal {
public:
    virtual void Say() const = 0;
};

template <typename T>
void Greet(const T &t) {
    t.Say();
}

class Mammal: public Animal {
public:
    void Say() const {
        std::cout << "Hello! I'm a mammal." << std::endl;
    }
};

class Reptile: public Animal {
public:
    void Say() const {
        std::cout << "Hello! I'm a reptile." << std::endl;
    }
};

int main() {
    Mammal m;
    Reptile r;
    std::vector<Animal> animals = {m, r};
    std::for_each(animals.begin(), animals.end(), [](const Animal &animal) {
        animal.Say();
    });
    return 0;
}

If you compile this with your C++11 compiler clang++ -std=c++11 -stdlib=libc++ code.cpp, you might get an error such as:

error: allocating an object of abstract class type 'Animal'

But, we are never allocating the instance of type Animal right? This is time to get our understanding of Generic Programming better.

Terminology

Generic Programming works around these main terminologies:

Concept
Model
Refinement
Type Constraint

Concept is the core design or interface. Model is what conforms to the Concept and provides a implementation of the Concept. Refinement is like inheriting a Concept and adding more things to it. And finally Type Constraint deals with specifying what and what not a Type that wishes to use the Concept is allowed to do.

In C++, we can have a pure virtual base class, where the base class provides no implementation for any member function. Swift hipster might know this idea as a protocol.

Along with that we have something in Swift that C++ programmers are not used to. It’s the use of protocol for Type Constraint. With C++, the compiler implicitly adds the Type Constraint for us.

Let’s take a look at some examples.

This is a Concept:

class Animal {
    public:
    virtual void Say() const = 0;
};

This is a Model:

class Mammal: public Animal { ... }

This is Refinement:

class WalkingAnimal: public Animal {
    public:
    virtual void Walk() const = 0;
};

This is a Type Constraint:

protocol Equatable {
    func ==(lhs: Self, rhs: Self) -> Bool
}

In Swift we use the term protocol for both Concept and Type Constraint, and this could be a source of confusion to many. Whereas, in C++ we don’t have any way to design Type Constraint, which helps when writing code, but when the code fails to compile for some reaons it might get hard to understand why.

First let us try to understand the relationship between a Model and a Type Constraint. Let’s say we have a Model named Student as:

struct Student: Equatable {
    let studentId: Int
}

func ==(lhs: Student, rhs: Student) -> Bool {
    return lhs.studentId == rhs.studentId
}

Here Student conforms to the Type Constraint. So any function that is designed to with types Equatable can work with Student. But, we have to keep in mind that the Student is not a Model of Concept named Equatable.

Next let us explore the relationship between a Model and a Concept. If we have a generic stack implementation:

class Stack<T> {
    private var store: [T] = []

    func push(item: T) {
        store.append(item)
    }

    func pop() -> T? {
        if store.isEmpty {
            return nil
        }
        return store.removeLast()
    }
}

What could be the core Concept of a stack? If we take a look at it, we might say a stack is a data structure where you push items and later on pop them back. So, we can say the Concept of stack can be written as:

protocol Stack {
    typealias Item
    func push(item: Item)
    func pop() -> Item?
}

Now, we can Model an Array backed stack on this Concept as:

class ArrayStack<T>: Stack {
    private var store: [T] = []

    func push(item: T) {
        store.append(item)
    }

    func pop() -> T? {
        if store.isEmpty {
            return nil
        }
        return store.removeLast()
    }
}

Or, we can go crazy and write a weird stack, where you pop data filtered by some epoch timestamp that we provide at runtime.

class TimeStack<T>: Stack {
    private var store: [CFTimeInterval: T] = [:]
    var epoch = CACurrentMediaTime()

    func push(item: T) {
        store[CACurrentMediaTime()] = item
    }

    func pop() -> T? {
        for (time, value) in store {
            if time < epoch {
                return value
            }
        }

        return nil
    }
}

Solution

With our basics now clear, we can take a look again at the original problem. What was wrong with our Animal example at the top?

The problem is with both the implementations.

In the Swift implementation, we’ve designed our Animal as a Type Constraint, as can be observed with the func greet<T: Animal>, but then later we are trying to use it as Concept by doing things like class Mammal: Animal. And this is what compiler is complaining about.

With our C++ implementation, as our dear clang compiler was trying to inform us, the Animal is actually a Concept, but we are trying to use it as Type Constraint by doing things like std::vector<Animal>.

And hence both the usages are invalid.

That’s all for today. We shall explore the other Generic Programming terminologies later someday.

Have a nice day!

A little bit of slope makes up for a lot of y-intercept

Josh Thompson — Fri, 26 Jun 2015 03:00:00 GMT

The following is recounted on Quora, from a lecture by Stanford professor John Ousterhout (he’s in the Computer Science department):

Here’s today’s thought for the weekend. A little bit of slope makes up for a lot of Y-intercept.

[Laughter]

So at a mathematical level this is an obvious truism. You know if you have two lines, the red line and the blue line and the red line has a lower Y-intercept but a greater slope then eventually the red line will cross the blue line.

And if the Y-axis is something good, depending on your definition of something good, then I think most people would pick the red trajectory over the blue trajectory (..unless you think you’re going to die before you get to the crossing point).

[Laughter]

So in a mathematical sense it’s kind of obvious. But I didn’t really mean in a mathematical sense, I think this is a pretty good guideline for life also. What I mean is that how fast you learn is a lot more important than how much you know to begin with. So in general I say that people emphasize too much how much they know and not how fast they’re learning.

That’s good news for all of you people because you’re in Stanford and that means you learn really, really fast. This is a great advantage for you. Now let me give you some examples. The first example is: you shouldn’t be afraid to try new things even if you’re completely clueless about the area you’re going into. No need to be afraid about that. As long as you learn fast you’ll catch up and you’ll be fine.

For example I often hear conversations the first week of class where somebody will be bemoaning, “Oh so-and-so knows blah-blah-blah, how am I ever going to catch up to them?” Well, if you’re one of the people who knows blah-blah-blah it’s bad news for you because honestly everyone is going to catch up really quickly. Before you know it that advantage is going to be gone and if you aren’t learning too you’re going to be behind.

Another example is that a lot of people get stuck in ruts in their lives. They realize they’re in the wrong job for them. I have the wrong job or the wrong spouse or whatever…

[Laughter]

And they’re afraid to go off and try something new. Often they’re worried, I’m going to really look bad if I go..

[Laughter]

I’m kidding about the spouse. But, seriously people will be afraid to try some new thing because they’re worried they’ll look bad or will make a lot of rookie mistakes. But, I say, just go do it and focus on learning.

[Laughter]

Let me take the spouse out of the equation for now.

[Laughter]

Focus on the job.

Another example is hiring. Before I came back to academia a couple of years ago I was out doing startups. What I noticed is that when people hire they are almost always hire based on experience. They’re looking for somebody’s resume trying to find the person who has already done the job they want them to do three times over. That’s basically hiring based on Y-intercept.

Personally I don’t think that’s a very good way to hire. The people who are doing the same thing over and over again often get burnt out and typically the reason they’re doing the same thing over and over again is they’ve maxed out. They can’t do anything more than that. And, in fact, typically what happens when you level off is you level off slightly above your level of competence. So in fact you’re not actually doing the current job all that well.

So what I would always hire on is based on aptitude, not on experience. You know, is this person ready to do the job? They may never have done it before and have no experience in this area, but are they a smart person who can figure things out? Are they a quick learner? And I’ve found that’s a much better way to get really effective people.

So I think this is a really interesting concept you can apply in a lot of different ways. And the key thing here I think is that slow and steady is great. You don’t have to do anything heroic. You know the difference in slopes doesn’t have to be that great if you just every day think about learning a little bit more and getting a little bit better, lots of small steps, its amazing how quickly you can catch up and become a real expert in the field.

I often ask myself: have I learned one new thing today? Now you guys are younger and, you know, your slope is a little bit higher than mine and so you can learn 2 or 3 or 4 new things a day. But if you just think about your slope and don’t worry about where you start out you’ll end up some place nice.

Ok, that’s my weekend thought.

[Applause]

Broken Build

Craig Pardey — Fri, 26 Jun 2015 03:00:00 GMT

I have used automated build tools on all my projects since 2000. Back then it was CruiseControl in all its XML-configuration glory. Today it is usually Jenkins, or sometimes TeamCity, and I recently tried out ThoughtWorks Go. The user interfaces have generally improved over time, and now there is probably a “plugin for that” thing you used to script manually, but overall not much has changed in the last 15 years.

In fact, I feel that The Build is actually flakier now than it was then, mostly due to the increased complexity.

The Poor Man's Postmortem - Lemma

Evan Todd — Fri, 26 Jun 2015 00:00:00 GMT

The big secret of our industry is, we don't actually enjoy making games. We slave away in obscurity for years in anticipation of one glorious day.

Not release day, no. The day we can finally write a postmortem full of pretentious anecdotes, bad jokes, and unsolicited advice.

Well I just finished a game, and doggone it, I am going to exercise my inalienable rights as a developer.

Lemma is immersive first-person parkour in a surreal, physics-driven voxel world.

Things to do when making a game

Ancient gamedev postmortem traditions mandate that this section be titled "what went right". Unfortunately, the game was so shockingly good and so many things went right that a full overview would stretch on endlessly.

TTY Week

Venam's Blog — Patrick Louis (Lebanon) — Fri, 26 Jun 2015 00:00:00 GMT

Hello fellow readers, This week is TTY week at nixers.net. Last summer we did the same challenge and it turned out a pretty enjoyable and a great learning experience. I'm going to add logs in this post and hopefully it'll render beautifully (because I can't check my blog from the TTY.)

The Gunslinger (The Dark Tower, #1)

Apurva Shukla's RSS Feed — Thu, 25 Jun 2015 14:02:21 GMT

⭐ ⭐ ⭐ ⭐ Slowly building momentum in order to set the epic series! Whilst starting as bland, King’s writing style instantly kicked up as his…

Bird atop Twin Peaks

june.kim — Thu, 25 Jun 2015 03:00:00 GMT

Anvil: Double O Seven

zserge's blog — Thu, 25 Jun 2015 03:00:00 GMT

Anvil: version 0.0.7 released! If you missed the news - Anvil is a tiny reactive UI library for Android. It’s based on the concept of the virtual layout (much like React.js) and brings declarative layouts with data bindings, componentization, styles, animations etc in pure Java. It also supports Kotlin, which is a great language and I hope it will replace Java on Android some day. Version 0.0.6 was suprisingly stable, but life goes on, and here’s version 0.

Windows scheduled task to auto logoff

Allan's Blog — Wed, 24 Jun 2015 13:52:36 GMT

Here's some PowerShell to remotely schedule a task to log out 2+ hours idle users:

function InstallAutoLogoff($computer)
{
    invoke-command -computer $computer `
    {
        schtasks.exe /delete /tn AutoLogOff /f 2>&1 | Out-Null

        set-content c:\windows\temp\AutoLogOffTask.xml `
        '<?xml version="1.0" encoding="UTF-16"?>
        <Task version="1.2" xmlns="http://schemas.microsoft.com/windows/2004/02/mit/task">
          <Triggers>
            <TimeTrigger>
              <Repetition>
                <Interval>PT30M</Interval>
              </Repetition>
              <StartBoundary>2015-01-01T00:00:00</StartBoundary>
            </TimeTrigger>
          </Triggers>
          <Actions Context="Author">
            <Exec>
              <Command>PowerShell.exe</Command>
              <Arguments>-Command $r1 = quser.exe; $idPos = $r1[0].IndexOf(''ID  ''); foreach($row in $r1) { $r2 = $row.Substring($idPos).Trim() -split ''  +''; if($r2[2].Contains('':'')) { $t = $r2[2].Replace('':'', ''.''); if($t.Contains(''+'') -or ($t -as [double] -ge 2)) { rwinsta.exe $r2[0] } } }</Arguments>
            </Exec>
          </Actions>
        </Task>'

        schtasks.exe /create /tn AutoLogOff /ru system /xml c:\windows\temp\AutoLogOffTask.xml
        del c:\windows\temp\AutoLogOffTask.xml
    }
}

InstallAutoLogoff 'myserver1'
InstallAutoLogoff 'myserver2'
InstallAutoLogoff 'myserver3'

The Little Prince

june.kim — Mon, 22 Jun 2015 03:00:00 GMT

Leather Tripod Chair

june.kim — Mon, 22 Jun 2015 03:00:00 GMT

JNR-FUSE library for using FUSE from Java

SerCe's blog — Mon, 22 Jun 2015 03:00:00 GMT

This article explains how to implement a userspace file system using Java, connecting Java and native code without writing C code, to achieve maximum performance.

Adventures after Google I/O-2(iOS).

Prasham H Trivedi — Mon, 22 Jun 2015 03:00:00 GMT

After trying and playing some new stuffs(except M preview) released in Google I/O, I have found some trivial things to take care of. In this part, we will see how should we take care if we add iOS details in our google cloud console and how it can affect android development.

Portugal travel bookmarks

xenodium.com @alvaro — Sun, 21 Jun 2015 03:00:00 GMT

As a Dermatologist, what life changing skin care advice would you give me?.
Avenida dos Platanos.
Beautiful Lisbon.
Best restaurants in Alghero?.
Boca do inferno.
Cabo da Roca.
Casa dos passarinhos (Steak on hot stone, tuna steak with “mirandesa” sauce, monkfish masada).
Cascais - Guincho.
Cervejaria Ramiro (seafood restaurant in town).
Eduard 7th park.
Estoril.
Fox Trot (bar).
Hachijō-jima island.
Ilha da Culatra (good for families).
Ilha da Tavira (good for food @ Portas do Mar).
Ilha Deserta (seafood as Estaminé).
Jeronimos Monastery (Lisbon).
Lisbon Castle.
Mercado da Ribeira in Lisbon.
Mouro's castle (Sintra).
Mouro's castle.
Pasteis de Belem (Lisbon).
Pastelaria piriquita eat queijadas de sintra.
Pasteleria Piriquita (Sintra).
Pavilhao Chines (bar).
Pena's Pallace (must see if you go to sintra)
Pena's Pallace (Sintra).
Pensao do Amor (bar).
Portugal Archives - ET Food Voyage.
Praça do comercio.
Quinta da Regaleira
Quinta da Regaleira (Sintra).
Sintra village
Sintra.
Stop do bairro (tamboril rice, prawn curry, and seafood rice).
Surfcastle.
São Bento railway station.
Sé (Lisbon Cathedral).
The best beaches in Portugal's Algarve.
Torre Belem (Lisbon).
Web Summit Ladies Craft Night - Tote Bag Embroidery Workshop Tickets, Tue, Nov 6, 2018 at 7:00 PM (Eventbrite).
What tips would you give to a newbie who wants to travel like you? - Quora.
World's longest pedestrian suspension bridge.

Bulgaria travel bookmarks

xenodium.com @alvaro — Sun, 21 Jun 2015 03:00:00 GMT

Devetashka Cave.

Presenting bookmarks

xenodium.com @alvaro — Sun, 21 Jun 2015 03:00:00 GMT

Bali travel bookmarks

xenodium.com @alvaro — Sun, 21 Jun 2015 03:00:00 GMT

Gili Islands travel (Nusa Tenggara, Indonesia - Lonely Planet).
Manta snorkeling Nusa Penida – When? Where? How much?.
Pura Lempuyang, Bali.

Changing the gears to Generic Programming

Whacky Labs — Sat, 20 Jun 2015 09:02:54 GMT

Practical usage of generic programming can be narrowed down to two forms:

Writing top level generic functions
Writing generic custom types

Generic Functions

Writing top level generic functions is easier. Its more of like you write a normal function and later on you realize that this function is not restricted to any type, rather it is just an abstract algorithm that can be used by many types, you upgrade it to generic type. For example,

bool isEqual(int x, int y) {
    return x == y;
}

can be easily abstracted out to a generic form such as

template <typename T>
bool isEqual(T x, T y) {
    return x == y;
}

This generic form would work as long as T supports a == operation. Which brings us to the term you will often find with generic programming, Type Constraints.

So, the above example would work fine with all types that support == operation. Such as:

std::cout << (isEqual(10, 10) ? "Y" : "N") << std::endl;
std::cout << (isEqual(10.5, 10.5) ? "Y" : "N") << std::endl;
std::cout << (isEqual("hello", "hello") ? "Y" : "N") << std::endl;

But would easily break down when used like:

struct Vector2 {
    float x, y;
    Vector2(float xx, float yy) : x(xx), y(yy) {}
};

std::cout << (isEqual(Vector2(10, 10), Vector2(10, 10)) ? "Y" : "N") << std::endl; // output compilation failure

And the reason being, that the Type Constraint has been violated. The type Vector2 does not provides an == operation. The fix is simple, satisfy the Type Constraint

bool operator == (const Vector2 &lhs, const Vector2 &rhs) {
    return (lhs.x == rhs.x) && (lhs.y == rhs.y);
}

Another type of error that could arise with generic functions is, where the constraints are not violated syntactically, by are logically broken. For example:

const char w1[] = "world";
const char w2[] = "world";
std::cout << (isEqual(w1, w2) ? "Y" : "N") << std::endl; // output N

Here, our isEqual() sees that the types are const char * and the compiler knows how to compare two pointers. But the result might or might not be what we were expecting. If you were expecting a string comparison, you’ve to tell the compiler how to do that.

For our specific case, it can be achieved by providing an overloaded implementation that just takes const char *

bool isEqual(const char *x, const char *y) {
    return strcmp(x, y) == 0;
}

Generic Types

The main power of generic programming lies when working with generic types. Just as generic functions abstract the algorithm to a type-free level, generic types abstract our design to type free level.

Think of generic types as if us providing the type information for the compiler, and passing the responsibility of creating the actual types to the compiler.

To get an quick overview, the Vector2 type that we wrote above for float types can easily be rewritten as a generic type

template <typename T>
struct Vector2 {
    T x, y;
    Vector2(T xx, T yy) : x(xx), y(yy) {}
};

And the == implementation should just fit with the change:

template <typename T>
bool operator == (const Vector2<T> &lhs, const Vector2<T> &rhs) {
    return (lhs.x == rhs.x) && (lhs.y == rhs.y);
}

Now, where should we actually use the generic types in real world? To address this question, you’ve to switch you mindset from run-time polymorphism to compile-time polymorphism. Let’s consider an example where you have designed your types in the classical object-oriented fashion, that is using inheritence

class Shape {
public:
    virtual ~Shape() {}
    void PrintArea() {
        std::cout << "area = " << Area() << std::endl;
    }

private:
    virtual float Area() const = 0;
};

class Rectangle: public Shape {
public:
    Rectangle(float w, float h) : width(w), height(h) {}

private:
    float Area() const {
        return width * height;
    }

    float width, height;
};

class Circle: public Shape {
public:
    Circle(float r) : radius(r) {}

private:
    float Area() const {
        return 3.14 * (radius * radius);
    }

    float radius;
};

int main() {
    Rectangle *r = new Rectangle(10, 20);
    Circle *c = new Circle(5);

    std::vector<Shape *> shapes;

    shapes.push_back(r);
    shapes.push_back(c);

    std::for_each(shapes.begin(), shapes.end(), [](Shape *s) {
        s->PrintArea();
    });

    delete r;
    delete c;

    return 0;
}

Here, we have a base Shape class that does provides the public interface and leaves the actual implementation details to the subclasses. In this case the actual calculation of the area. Internally, we’re calling the Area() function on Shape and let the run-time do the actual look-up for the implementation by looking at the type information.

In another words, what we are actually doing is that delegating some part of the implementation of our Shape class to subclasses. Another way of achieving the same results is by not depending on subclasses for overrides, but delegating the implementation to some external classes.

struct ShapeImpl {
    virtual float Area() const = 0;
};

class RectangleImpl: public ShapeImpl {
public:
    RectangleImpl(float w, float h) : width(w), height(h) {}

private:
    float Area() const {
        return width * height;
    }

    float width, height;
};

class CircleImpl: public ShapeImpl {
public:
    CircleImpl(float r) : radius(r) {}

private:
    float Area() const {
        return 3.14 * (radius * radius);
    }

    float radius;
};

class Shape {
public:
    Shape(const ShapeImpl *impl) : implementation(impl) {}

    void PrintArea() const {
        std::cout << "area = " << implementation->Area() << std::endl;
    }

private:
    const ShapeImpl *implementation;
};

int main() {
    RectangleImpl *r = new RectangleImpl(10, 20);
    CircleImpl *c = new CircleImpl(5);

    std::vector<Shape> shapes;

    shapes.push_back(Shape(r));
    shapes.push_back(Shape(c));

    std::for_each(shapes.begin(), shapes.end(), [](const Shape &s) {
        s.PrintArea();
    });

    delete r;
    delete c;

    return 0;
}

I don’t know if we’ve achieved any actual improvement over the earlier implementation with this or not. But, this gives you an idea that the implementation can be dragged out of the core class while keeping the visible interface of the Shape intact.

If we can achieve this, we can even go a step further and actually provide the implementation at compile time. In other words, we can simply make the Shape class generic, and let the compiler plug-in the implementation details at compile-time.

template <typename Impl>
class Shape {
public:
    Shape(const Impl impl) : implementation(impl) {}

    void PrintArea() const {
        std::cout << "area = " << implementation.Area() << std::endl;
    }

private:
    const Impl implementation;
};

Now, we can create and use our generic shape instances as:

    Shape<RectangleImpl> rect(RectangleImpl(10, 20));
    Shape<CircleImpl> circle(CircleImpl(5));

    rect.PrintArea();
    circle.PrintArea();

The only type constraints on the RectangleImpl and CircleImpl is that they need to have a Area() function.

class RectangleImpl {
public:
    RectangleImpl(float w, float h) : width(w), height(h) {}

    float Area() const {
        return width * height;
    }

private:
    float width, height;
};

class CircleImpl {
public:
    CircleImpl(float r) : radius(r) {}

    float Area() const {
        return 3.14 * (radius * radius);
    }

private:
    float radius;
};

The C++ compiler is smart enough to see the constraints are not violated by any new delegate class. For example if we introduce a TriangleImpl as:

class TriangleImpl {
public:
    TriangleImpl(float b, float h) : base(b), height(h) {}

private:
    float base, height;
};

Shape<TriangleImpl> triangle(TriangleImpl(10, 5));
triangle.PrintArea();

The compiler will immediately throw error messages, unless you fix your implementation by providing the Area() function.

class TriangleImpl {

public:
    TriangleImpl(float b, float h) : base(b), height(h) {}

    float Area() const {
        return 0.5 * base * height;
    }

private:
    float base, height;
};

Now, if you notice that we are not using an std::vector anymore, that is because we don’t have a Shape type anymore. And if you find yourself thinking in this direction, you need to get out of the run-time polymorphic mindset. When using compile-time polymorphism you’ve to keep in mind the fact that now you’re not in control of creating the custom types, rather you’ve passed on that authority to the compiler.

So, Shape<TriangleImpl> and Shape<CircleImpl> are entirely different types with no common ancestor or anything.

When designing with compile time polymorphism you need to keep in mind that your generic type should be a leaf node or a final class in the entire type system. You should not have to subclass your generic type, rather plug-in a delegate class that provides the implementation that you would rather provide by sub-classing.

To summarize, here’s a quick list of good and bad of generic type system:

Pro

Robust: No run time exceptions.
Efficient: No run time lookups

Cons

Not a good candidate for base class
Hard to read and debug.

Now, moving on to using generics with Swift. Swift provides the same good old C++ way of writing generic code with some extra type constraint system. Where in C++ the compiler would do the type constraint checking when you actually compile the code, Swift actually makes the constraint system more explicit, such that you have to provide the constraint information with a protocol.

protocol ShapeImplementable {
    var area: Double { get }
}

struct Shape<Impl: ShapeImplementable> {
    private let implementation: ShapeImplementable

    init(impl: ShapeImplementable) {
        implementation = impl
    }

    func printArea() {
        println("area = \(implementation.area)")
    }
}

And then you can implement your delegate class as

struct RectangleImpl: ShapeImplementable {
    var area: Double {
        return width * height
    }

    init(w: Double, h: Double) {
        width = w
        height = h
    }

    private let width: Double
    private let height: Double
}

And finally use them the same way you would with C++.

let rect = Shape<RectangleImpl>(impl: RectangleImpl(w: 10, h: 20))

The entire code for this article is also available at github.com/chunkyguy/GenericsDemo

Goodbye and have a nice day!

Optimization with Timeit

denvaar's website — Sat, 20 Jun 2015 08:40:09 GMT

Code optimization analysis using Python's timeit module.

2015-06-20

ho.dges.online — Sat, 20 Jun 2015 03:00:00 GMT

The Girl on the Train

Apurva Shukla's RSS Feed — Fri, 19 Jun 2015 10:00:00 GMT

⭐ ⭐ ⭐ 3.5/5 Fast-paced, thrilling, and creepy. What else could you want? Well, for one - likeable characters would be a move in the right…

Old ventures and new adventures

Tanel Poder Blog — Fri, 19 Jun 2015 09:08:33 GMT

I have some news, two items actually.

First, today (it’s still 18th June in California) is my blog’s 8th anniversary!

I wrote my first blog post, about Advanced Oracle Troubleshooting, exactly 8 years ago, on 18th June 2007 and have written 229 blog posts since. I had started writing and accumulating my TPT script collection a couple of years earlier and now it has over 1000 files in it! And no, I don’t remember what all of them do and even why I had written them. Also I haven’t yet created an index/documentation for all of them (maybe on the 10th anniversary? ;)

My love-hate relationship with LuaJIT

Separate Concerns — Thu, 18 Jun 2015 03:00:00 GMT

Clarification: The title is exaggerated. I have never hated LuaJIT, I just went back to using PUC Lua primarily.

I started using Lua in early 2007. I had already been programming for years, but I had reached a point where I had decided that I wanted to know exactly what was going on when my computer executed dynamic code. That meant understanding two critical pieces of the software stack: the OS and the interpreter.

I already had notions of how an interpreter works and I could tell the main functions of a Unix kernel, but really understanding software, for me, implied reading its source code. I quickly realized that I would not succeed if I started with Perl or Python and Linux, which were what I was using at the time, so I set my sights on Lua and Minix3.

Knowing software also implies using it, so Minix3 and Lua became my main OS and language for several months. To be honest, I was running Minix3 in a VM due to the lack of drivers for my hardware. I started to study their code, with the help of the book for Minix3 and resources found online for Lua. I learnt a lot during that period.

Eventually, I went back to Linux for practical reasons, but I had been charmed by the down-to-Earth elegance of Lua. It became my favorite programming language. I was still a student though, so I mostly wrote small utilities, Web pages and game prototypes with it. Coursework was Java, Python and C, and for “serious” stuff at ResEl I used Perl, Python or Bash.

Then in 2010 I went to Moodstocks for my MSc Thesis. We were a very tiny startup then, just the founders and a couple of interns. They were a Ruby and C++ shop, so I learnt Ruby, but I didn’t like it: too much magic, and the object-oriented interface to write bindings was a mess compared to the very clean Lua C API.

Eventually, Moodstocks hired me and I worked for them for over three years. I ended up writing most of their server-side code, I already gave details here. A lot of that code leveraged LuaJIT. It was the perfect tool for the job: as flexible and dynamic as Ruby so it worked well as a scripting language, about as fast as C++ so it could be used to implement Computer Vision algorithms, and the FFI made it very easy to call into our existing C / C++ libraries.

Those are the features that make LuaJIT so attractive: its blazing speed and the FFI. They are why I could write Lua professionally for several years, and I could never thank Mike Pall enough for this. Moreover, LuaJIT has been playing a huge role in the growing popularity of Lua those last few years.

However, paradoxically, LuaJIT negates the very reason that made me pick Lua in the first place. And by that I mean: I do not deeply understand how LuaJIT works. I have tried, and I will certainly try again, but it is one or two orders of magnitude more complicated than PUC Lua.

Beyond the purist ideal of understanding the whole stack, this has practical implications. When you find a bug in LuaJIT, understanding it and fixing it is terribly complicated. The best I can do is usually to try to produce a small test case that reproduces the bug (even that is not always easy) and hope Mike Pall finds a fix. With PUC Lua I could probably fix it myself - but of course PUC Lua is so simple that I have never found a bug in it!

Another issue is that PUC Lua and LuaJIT are diverging. LuaJIT implements Lua 5.1. Lua 5.2 code can be made to work as long as it does not use _ENV, but code that leverages the new features in Lua 5.3 is not supported at all (although modules like lua-compat-5.3 can probably help). On the other hand, more and more modules require the FFI, making them incompatible with PUC Lua (of course I am a culprit of that myself). And to top it all, the interpreters work so differently that efficient code in LuaJIT is not necessarily good in PUC Lua, and vice versa.

These days, the purist, simplicity-loving part of me tends to win over the pragmatist in my (rare) non-work code, and my current target of choice is PUC Lua. At work, I write almost exclusively C and almost no Lua, and the little I do write runs on a machine which does not support LuaJIT. The only reasons I still use LuaJIT are maintenance of luajit-msgpack-pure and my use of OpenResty, which for my purposes could as well be built with PUC Lua instead.

That being said, I like knowing that I could take my Lua code and, with a few tweaks and a change of interpreter, get a serious performance boost. So I still hope someone (if not Mike Pall) will make a version of LuaJIT that supports the Lua 5.3 interface. Otherwise, I guess I can still decide that learning more about JIT compilation is worth it and spend a few months diving into that LuaJIT codebase!

Oh, and by the way, with this whole systemd thing, I am semi-seriously considering giving Minix3 a second chance :)

Attenuated Transmission

Code & Visuals — Thu, 18 Jun 2015 03:00:00 GMT

A few months ago I added attenuation to Takua a0.5’s Fresnel refraction BSDF. Adding attenuation wound up being more complex than originally anticipated because handling attenuation through refractive/transmissive mediums requires volumetric information in addition to the simple surface differential geometry. In a previous post about my BSDF system, I mentioned that the BSDF system only considered surface differential geometry information; adding attenuation meant extending my BSDF system to also consider volume properties and track more information about previous ray hits.

First off, what is attenuation? Within the context of rendering and light transport, attenuation is when light is progressively absorbed within a medium, which results in a decrease in light intensity as one goes further and further into a medium away from a light source. One simple example is in deep water- near the surface, most of the light that has entered the water remains unabsorbed, and so the light intensity is fairly high and the water is fairly clear. Going deeper and deeper into the water though, more and more light is absorbed and the water becomes darker and darker. Clear objects gain color when light is attenuated at different rates according to different wavelengths. Combined with scattering, attenuation is a major contributing property to the look of transmissive/refractive materials in real life.

Attenuation is described using the Beer-Lambert Law. The part of the Beer-Lambert Law we are concerned with is the definition of transmittance:

\[ T = \frac{\Phi_{e}^{t}}{\Phi_{e}^{i}} = e^{-\tau}\]

The above equation states that the transmittance of a material is equal to the transmitted radiant flux over the received radiant flux, which in turn is equal to e raised to the power of the negative of the optical depth. If we assume uniform attenuation within a medium, the Beer-Lambert law can be expressed in terms of an attenuation coefficient μ as:

\[ T = e^{-\mu\ell} \]

From these expressions, we can see that light is absorbed exponentially as distance into an absorbing medium increases. Returning back to building a BSDF system, supporting attenuation therefore means having to know not just the current intersection point and differential geometry, but also the distance a ray has traveled since the previous intersection point. Also, if the medium’s attenuation rate is not constant, then an attenuating BSDF not only needs to know the distance since the previous intersection point, but also has to sample along the incoming ray at some stepping increment and calculate the attenuation at each step. In other words, supporting attenuation required BSDFs to know the previous hit point in addition to the current one and also requires BSDFs to be able to ray march from the previous hit point to the current one.

Adding previous hit information and ray march support to my BSDF system was a very straightforward task. I also added volumetric data support to Takua, allowing for the following attenuation test with a glass Stanford Dragon filled with a checkerboard red and blue medium. The red and blue medium is ray marched through to calculate the total attenuation. Note how the thinner parts of the dragon allow more light through resulting in a lighter appearance, while thicker parts of the dragon attenuate more light resulting in a darker appearance. Also note the interesting red and blue caustics below the dragon:

Things got much more complicated once I added support for what I call “deep attenuation”- that is, attenuation through multiple mediums embedded inside of each other. A simple example is an ice cube floating in a glass of liquid, which one might model in the following way:

There are two things in the above diagram that make deep attenuation difficult to implement. First, note that the ice cube is modeled without a corresponding void in the liquid- that is, a ray path that travels through the ice cube records a sequence of intersection events that goes something like “enter water, enter ice cube, exit ice cube, exit water”, as opposed to “enter water, exit water, enter ice cube, exit ice cube, enter water, exit water”. Second, note that the liquid boundary is actually slightly inside of the inner wall of the glass. Intuitively, this may seem like a mistake or an odd property, but this is actually the correct way to model a liquid-glass interface in computer graphics- see this article and this other article for details on why.

So why do these two cases complicate things? As a ray enters each new medium, we need to know what medium the ray is in so that we can execute the appropriate BSDF and get the correct attenuation for that medium. We can only evaluate the attenuation once the ray exits the medium, since attenuation is dependent on how far through the medium the ray traveled. The easy solution is to simply remember what the BSDF is when a ray enters a medium, and then use the remembered BSDF to evaluate attenuation upon the next intersection. For example, imagine the following sequence of intersections:

Intersect glass upon entering glass.
Intersect glass upon exiting glass.
Intersect water upon entering water.
Intersect water upon exiting water.

This sequence of intersections is easy to evaluate. The evaluation would go something like:

Enter glass. Store glass BSDF.
Exit glass. Evaluate attenuation from stored glass BSDF.
Enter water. Store water BSDF.
Exit water. Evaluate attenuation from stored water BSDF.

So far so good. However, remember that in the first case, sometimes we might not have a surface intersection to mark that we’ve exited one medium before entering another. The following scenario demonstrates how this first case results in missed attenuation evaluations:

Intersect water upon entering water.
Exit water, but no intersection!
Intersect ice upon entering ice.
Intersect ice upon exiting ice.
Enter water again, but no intersection either!
Intersect water upon exiting water.

The evaluation sequence ends up playing out okay:

Enter water. Store water BSDF.
Exit water, but no intersection. No BSDF evaluated.
Enter ice. Intersection occurs, so evaluate attenuation from stored water BSDF. Store ice BSDF.
Exit ice. Evaluate attenuation from stored ice BSDF.
Enter water again, but no intersection, so no BSDF stored.
Exit water…. but there is no previous BSDF stored! No attenuation is evaluated!

Alternatively, in step 6, instead of no previous BSDF, we might still have the ice BSDF stored and evaluate attenuation based on the ice. However, this result is still wrong, since we’re now using the ice BSDF for the water.

One simple solution to this problem is to keep a stack of previously seen BSDFs with each ray instead of just storing the previously seen BSDF. When the ray enters a medium through an intersection, we push a BSDF onto the stack. When the ray exits a medium through an intersection, we evaluate whatever BSDF is on the top of the stack and pop the stack. Keeping a stack works well for the previous example case:

Enter water. Push water BSDF on stack.
Exit water, but no intersection. No BSDF evaluated.
Enter ice. Intersection occurs, so evaluate water BSDF from top of stack. Push ice BSDF on stack.
Exit ice. Evaluate ice BSDF from top of stack. Pop ice BSDF off stack.
Enter water again, but no intersection, so no BSDF stored.
Exit water. Intersection occurs, so evaluate water BSDF from top of stack. Pop ice BSDF off stack.

Excellent, we now have evaluated different medium attenuations in the correct order, haven’t missed any evaluations or used the wrong BSDF for a medium, and as we exit the water and ice our stack is now empty as it should be. The first case from above is now solved… what happens with the second case though? Imagine the following sequence of intersections where the liquid boundary is inside of the two glass boundaries:

Intersect glass upon entering glass.
Intersect water upon entering water.
Intersect glass upon exiting glass.
Intersect water upon exiting water.

The evaluation sequence using a stack is:

Enter glass. Push glass BSDF on stack.
Enter water. Evaluate glass attenuation from top of stack. Push water BSDF.
Exit glass. Evaluate water attenuation from top of stack, pop water BSDF.
Exit water. Evaluate glass attenuation from top of stack, pop glass BSDF.

The evaluation sequence is once again in the wrong order- we just used the glass attenuation when we were traveling through water at the end! Solving this second case requires a modification to our stack based scheme. Instead of popping the top of the stack every time we exit a medium, we should scan the stack from the top down and pop the first instance of a BSDF matching the BSDF of the surface we just exited through. This modified stack results in:

Enter glass. Push glass BSDF on stack.
Enter water. Evaluate glass attenuation from top of stack. Push water BSDF.
Exit glass. Evaluate water attenuation from top of stack. Scan stack and find first glass BSDF matching the current surface’s glass BSDF and pop that BSDF.
Exit water. Evaluate water attenuation from top of stack. Scan stack and pop first matching water BSDF.

At this point, I should mention that pushing/popping onto the stack should only occur when a ray travels through a surface. When the ray simply reflects off of a surface, an intersection has occurred and therefore attenuation from the top of the stack should still be evaluated, but the stack itself should not be modified. This way, we can support diffuse inter-reflections inside of an attenuating medium and get the correct diffuse inter-reflection with attenuation between diffuse bounces! Using this modified stack scheme for attenuation evaluation, we can now correctly handle all deep attenuation cases and embed as many attenuating mediums in each other as we could possibly want.

…or at least, I think so. I plan on running more tests before conclusively deciding this all works. So there may be a followup to this post later if I have more findings.

A while back, I wrote a PIC/FLIP fluid simulator. However, at the time, Takua Renderer didn’t have attenuation support, so I wound up rendering my simulations with Vray. Now that Takua a0.5 has robust deep attenuation support, I went back and used some frames from my fluid simulator as tests. The image at the top of this post is a simulation frame from my fluid simulator, rendered entirely with Takua a0.5. The water is set to attenuate red and green light more than blue light, resulting in the blue appearance of the water. In addition, the glass has a slight amount of hazy green attenuation too, much like real aquarium glass. As a result, the glass looks greenish from the ends of each glass plate, but is clear when looking through each plate, again much like real glass. Here are two more renders:

WWDC app for OS X

xenodium.com @alvaro — Wed, 17 Jun 2015 03:00:00 GMT

Guilherme Rambo created a great OS X OS X app for viewing WWDC content. Just installed it. Super handy. Thanks. Installing as simple as:

$ brew cask install wwdc

SSD Storage Cluster - Update and Diagram

smcleod.net — Wed, 17 Jun 2015 03:00:00 GMT

Due to several recent events beyond my control I’m a bit behind on the project - hence the lack of updates which I apologise for.

The goods news is that I’m back working to finish off the clusters and I’m happy to report that all is going to plan.

Here is the final digram of the two-node cluster design:

Plain text version available here

This was generated from the LCMC tool (beware - it’s java!).

Trainwreck

🦉 olshansky 🦁 — Tue, 16 Jun 2015 09:31:24 GMT

Olshansky's review of Trainwreck

Catholics, Free Software and Free Knowledge, again

Welcome to Marco Fioretti's website! on Stop at Zona-M — Tue, 16 Jun 2015 09:21:37 GMT

Almost ten years ago, I wrote about Free Software’s surprising sympathy with Catholic doctrine, noting that, albeit certain statements sound _“as if they could have been written by Richard M. Stallman, founder of the Free Software Foundation (FSF), in fact, they come from the Vatican Report “Ethics in Internet” (EiI)”.

Give Yourself More Playtime

Martin Rue — Tue, 16 Jun 2015 02:00:00 GMT

I remember my first computer fondly. As a kid I was more than addicted to the Commodore 64 – perfectly happy to sit and endure at least 5 minutes of the most colourful, potentially epileptic attack you can imagine, simply to have a few games of Boulder Dash.

Debugging Objective-C reference cycles

xenodium.com @alvaro — Mon, 15 Jun 2015 03:00:00 GMT

Overriding retain/release/autorelease may be handy while debugging:

- (instancetype)retain {
  NSLog(@"%p, retain\n", self);
  return [super retain];
}

- (oneway void)release {
  NSLog(@"%p, release\n", self);
  [super release];
}

- (instancetype)autorelease {
  NSLog(@"%p, autorelease\n", self);
  return [super autorelease];
}

Leather Sunglasses Case

june.kim — Mon, 15 Jun 2015 03:00:00 GMT

FinTech Startups Tug the Industry Forward

Dan Stroot — Mon, 15 Jun 2015 03:00:00 GMT

Financial services companies are among the most highly regulated industries in the world. They also suffer from a trust issue. Trust is a key ingredient in making a financial services business viable. Why would I ever buy insurance, banking services or invest my savings via a company that I didn’t trust?

This post FinTech Startups Tug the Industry Forward first appeared on Dan Stroot's Blog

Hot Tub Time Machine 2

🦉 olshansky 🦁 — Sun, 14 Jun 2015 06:42:43 GMT

Olshansky's review of Hot Tub Time Machine 2

Kingsman: The Secret Service

🦉 olshansky 🦁 — Sun, 14 Jun 2015 06:33:45 GMT

Olshansky's review of Kingsman: The Secret Service

American History X

🦉 olshansky 🦁 — Sun, 14 Jun 2015 06:23:39 GMT

Olshansky's review of American History X

log - a drop-in replacement for android Log class

zserge's blog — Sun, 14 Jun 2015 03:00:00 GMT

Once upon a time I was making a pretty large app for Android. When everything was almost finished I wondered how shall I remove all my logs before publishing the app? Looks like many other developers are interested in the same question as well. Surprisingly, the recommented way out is to strip off the Log class from the production builds using ProGuard. Maybe Proguard is nice, but to me it’s really weird why instead of having a nice out-of-box API for logging I need to modify my byte code with some 3rd party tool?

Truss Resume Creator

Sam Solomon — Sun, 14 Jun 2015 03:00:00 GMT

Truss is a beautiful, simple tool for creating resumes.

Mad Max: Fury Road Review

Honest Musings — Sat, 13 Jun 2015 18:23:50 GMT

Dirt. Metal. Water. War…thump…thump…thump. Blood. Grime…Guitar. Drums…dum..dum…dum. Human.Inhuman.God.Monster. Hope…survive…survive…survive! It’s a metal rock show in an apocalyptic wasteland masquerading as a movie. And I am not saying that only because of the humongous guitar mounted on an armored truck. It starts with a far off beat when the protagonist is suddenly made aware of being … Continue reading

London grub

xenodium.com @alvaro — Sat, 13 Jun 2015 03:00:00 GMT

Beyond the hype, buzz, and pricey gimmicks… Places to eat in London:

Antipode.
Arang.
Bone Daddies.
Gelupo.
Grind.
Holy Cow.
Kerbisher and Malt.
Kulu Kulu (South Ken).
Le Relais de Venise.
Lucky 7's.
Royal China.
Shree Krishna Vada Pav.
Sri Suwoon.
Tayyabs.
The Cow.
Tonkotsu.

A super privileged Puppet container

The Technical Blog of James on purpleidea.com — Fri, 12 Jun 2015 21:06:55 GMT

In this new crazy world of containers and immutable hosts, one might still want to run previous generation software such as Puppet on a current generation Atomic host. This article will explain how you can do that, and offer some proof of concept code.

The atomic host doesn’t provide a yum or dnf command, because the software is pre-baked into a read-only /usr/ partition. To “install” (to use) additional software, it usually needs to be distributed and run as a container.

The Magic of Reality

Apurva Shukla's RSS Feed — Fri, 12 Jun 2015 10:00:00 GMT

⭐ ⭐ ⭐ ⭐ Reality is often mundane to us, so much so that we tend to forget the true beauty of life and our world. This short read really got…

The Essential Neruda

june.kim — Wed, 10 Jun 2015 03:00:00 GMT

Zen and the Art of Motorcycle Maintenance

june.kim — Wed, 10 Jun 2015 03:00:00 GMT

My working playlist

xenodium.com @alvaro — Mon, 08 Jun 2015 03:00:00 GMT

It's been a while since I spotted The Ultimate Music Collection for Getting Work Done. Since then, I've been on the lookout for music to work to. Some favorites:

Moving past averages in SQL (Postgres) – Percentiles

CRAIG KERSTIENS — Sun, 07 Jun 2015 23:55:56 GMT

Often when you’re tracking a metric for the first time you take a look at your average. For example what is your ARPU - Average Revenue Per User. In theory this tells you if you can acquire new user how much you’ll make off that user. Or maybe what’s your average life time value of a customer. Yet, many that are more familiar looking and extracting meaning from data median or a few different looks at percentiles can be much more meaningful.

And while you can very easily get the AVG in Postgres, with a small amount more effort you can report on percentiles as well. Window functions have been around for some time in Postgres. They allow you to order your result set over a certain group. The most basic example is if you want to order by date, but know which one falls at place 10 in order you can use a window function and project out the rank().

Beyond outputting the rank yourself and doing extra manipulation Postgres has some great utilities to make the most common uses even easier. Being able to compute things such as the perc 95 directly on the data, or lay out for every record in the result where it falls within a percentile is hugely useful. Let’s take a look:

Assuming you have a table called purchases, which has a total in it we could try:

SELECT id,
total,
ntile(100) OVER (ORDER BY total) AS perc_rank
FROM purchases

This would give us something like:

 id | total | perc_rank
----------|---------|-----------
264 | 12034 | 100
643 | 11830 | 100
...
...
304 | 751 | 95

What this would tell us is we have less than 5% of our purchases that have a total over 751. From here you can start to dig in and extract all sorts of different meanings, and by doing directly in SQL you’re closer to the data and have one less processing step.

Percentiles get even more fun with the ordered set functions that came out in Postgres 9.4. They even allow you to project out hypothetical values in certain cases. For now I’d encourage adding ntile to your toolbox anytime you’re analyzing average or medians it will make your world a bit better, and then consider exploring further on the ordered set functions

Denim Sling Bag

june.kim — Sun, 07 Jun 2015 03:00:00 GMT

2015-06-07/01

ho.dges.online — Sun, 07 Jun 2015 03:00:00 GMT

2015-06-07/02

ho.dges.online — Sun, 07 Jun 2015 03:00:00 GMT

Thoughts on Writing

Nemo's Home — Sun, 07 Jun 2015 03:00:00 GMT

I have always wanted to be a writer. I think secretly us all reader-folk have that ambition. The joy of getting across your thoughts to another person without ever having met them is enormous.

Most of my writing time these days is spent over email, chat or my not-so-frequent blog posts. I tend to do a lot of research while writing, and it takes up a lot of time. As such, my writing output tends to be diminutive compared to what I’d like.

However, if you’ll go through my blog posts and emails, I write a lot about trivial things. Things that many people have already written about. Things that have probably been discussed to death, and where I have very little chance of actually coming up with something new.

Should I still go ahead and write about it?

This question has been bugging me for a while, especially as a blogger. I mostly write on technical topics these days. For instance, I have given talks on [Software Development], [UX Design], and even [Bitcoin]. I am nowhere close to being an authority on any of these things. Even in my specialized field of Web Development, there are so many things that I’m only barely aware about. So many things I am yet to even form my own opinions about. Topics I don’t even know exist.

When I go and read an article about Software Development from Joel Spolsky, or an article on Security by Bruce Schneier, or something on Startups by Paul Graham, or tptacek on Hacker News; I instantly sit back and take notice: I know their credentials and the fact that they are speaking authoritatively on the topic. However, what can I, a meager undergrad with almost zero experience, write on such topics. Why should I even try, when there are people hundreds of time better who understand these things a thousand times better than me.

In retrospect, this sounds quite similar to the Imposter Syndrome; and I’m not sure if this is exactly the same thing. I don’t get a feeling that I’m a fraud. I totally understand my own capabilities and successes, but the mere fact that there are people far better at what I’m doing is enough to dis-hearten me.

I’ve given this a lot of thought. A really good summary of my response is in the following answer by James Erwin, author of Rome Sweet Rome in a reddit AMA to a question asking for writer advice:

And if you’re going to write, write what you want to write. The odds against any creator are insane. If you’re going to devote months of your time, don’t let it be for an idea you think will sell. Odds are it won’t. Write something you want to write, or need to write. Write for yourself before anyone else. I’d rather read someone who is excited and passionate about what they want to say than someone who’s obviously trying to say what they think I want to hear.

— James Erwin

I write, despite all these doubts, for the following reasons:

Self-learning. A blog is an excellent way to keep track of your self-learning. Its amazing to come back a few years later and see the things you were struggling with before. Its equally amazing to do a trivial google search for an issue you face and find your own blog post or stackoverflow answer on the same.
Sharing Knowledge. Yes, there are people who might know it better, but that shouldn’t mean I should keep my knowledge to myself. That would go against all the values that I stand for.
Network Effect: Not in the strictest sense of the word, but my friend Shashank recently brought this up. I have a circle of people who know me and would vouch for my credentials. For the same reason, they are more likely to trust me as source, instead of a third person who they have no knowledge of.
I love writing. The mere process of putting words down is enchanting for me.

The next question that rises is: “What should I write about?”. Ruling out things I have no clue about, that still leaves a large number of topics I can cover. I am interested in UX Design, Hackers, Computer Security, Software Development, rationalism, skepticism, Free and Open Source movements, Political activism, Technocracy with a passing interest in several other fields such as cosmology and geek culture.

I am not going to pick one every day and write about something new. I don’t want to write something rubbish just for the sake of writing it. I ultimately want to write because I have something to say. It doesn’t have to be unique or ground-breaking. What matters is that I want to write about it.

A few days back someone contacted me on facebook asking me advice on getting started with web development. I get a lot of these queries, mostly over facebook, email, and quora. Our conversation went back and forth with me suggesting resources, and he getting exceedingly confused over whether he should use codecademy or udacity, or coursera or something else.

I have devoted a lot of time in my life to teaching people the nuances of these things. I have mentored many people, and acutely know the issues a beginner faces. In turn, I had an amazing mentor who taught me the importance of always learning things.

All of this lead me to realize one fact: I have been writing a lot about Software Development. Unfortunately, a lot of it is in private emails and chat. And I wanna write more about it, on a public medium.

So, I’m announcing the next thing I’m working on: a book called The Joy of Software Development. A few obligatory links:

The source code is available on GitHub.
Its licensed under the CC-by-SA 4.0 license.
The canonical url for the site is https://josd.captnemo.in/
Its hosted on the excellent GitBook platform, which automatically publishes each version as epub, mobi, and pdf as well.
Who the target audience for the book is.
You can file an issue for critique on GitHub

As expected, all the development, writing, and discussion on the book will be in the public, mostly on GitHub. I am writing this book, because I feel it needs to be written. I don’t expect it to be published, but that won’t stop me from writing it.

Artur Siekielski recently came across it, and wrote the following:

The book you’re writing looks very good! It’s filling a niche as I don’t think there are any modern books that focus on “bird’s eye view”, and I see it would be helpful for many programmers to refresh knowledge.

That gave me a bit of validation, as the only person who’d read it so far were my close friends.

If you wanna support its development, you can do one of the following:

Poke me on twitter or email and let me know you want to read it
Subscribe to the mailing list (I’ll send out updates there)
Watch or Star the repo on GitHub.
See the CONTRIBUTING file on github for contributing to the text.

Asking for Donations might sound weird to some. I don’t really need the money, but I think I’d get an additional sense of responsibility towards finishing it if people start giving me money. I will be donating the entire proceedings to EFF.

Monolith vs Microservices

Nicola Iarocci — Thu, 04 Jun 2015 03:00:00 GMT

I spent the last couple days in Paris with a bunch of smart architects and developers who are about to deal with a complex Monolithic-to-Microservices architecture transition.

On my flight back to Italy I jotted down a few thoughts on the topic and then, this morning, the first thing that comes up on my newsfeed is Monolith First by Martin Fowler. Stunning, because the essay content totally resonates with my own notes, so much that it would perfectly serve as a recap for them.

Adventures after Google I/O-1(Snackbar).

Prasham H Trivedi — Thu, 04 Jun 2015 03:00:00 GMT

After trying and playing some new stuffs(except M preview) released in Google I/O, I have found some trivial things to take care of. In the first part of the series I will talk about Snackbars.

Building an Object Graph in Rails

set_trace_func — Thu, 04 Jun 2015 03:00:00 GMT

I was needing to do some object cleanup in our rails app the other day, and purge some malformed objects, so I put together a quick script using some ActiveRecord reflection to walk the object chain.

# Squelch SQL logs, if you're running from rails console
ActiveRecord::Base.logger.level = 1

# Output formatters. Trailing ':' on both, plus the staggered indent
# of 4n and 4n+2 makes the output valid YAML, if automated analysis
# is called for.
def puts_node(node, indent)
  puts "    "*indent + node.class.name + "#" + node.id.to_s + ":"
end
def puts_assoc(assoc, indent)
  puts "    "*indent + "  " + assoc.to_s + ":"
end

def puts_tree(node, seen=[], indent=0)
  puts_node(node, indent)

  unless seen.include? node
    # seen maintains a list of nodes to avoid mutual recursion
    seen << node
    node.class.reflections.keys.each do |assoc|
      # To see all locations an object is referenced, get rid of the
      # "- seen" here. I only cared about which objects were present
      # anywhere in the tree, so this was fine.
      associated = Array(node.send(assoc)) - seen
      next if associated.empty?

      # Print an entry for the association, then recurse
      puts_assoc(assoc, indent)
      associated.each do |subnode|
        puts_tree(subnode, seen, indent+1)
      end
    end
  end

  # Also outputs a footer listing all seen objects once, take it or leave it
  if indent.zero?
    puts
    seen.each {|node| puts_node(node, indent)}
  end

  nil # return nil to avoid flooding terminal in rails console
end

# Usage: call with the root node for the object graph
puts_tree(User.find(42))

Worked like a charm, and made it easy to compare my bad object with other good ones. Just be careful of any global objects (a common shared subscription package, for instance, that has_many :users) that could lead to traversing your entire database, or logging associations that could overwhelm your output on older or heavily used objects. Subtracting a blacklist from reflection keys on line 18 would do the trick there.

When is migrating to cloud not the best choice?

Daniel Sada Caraveo | Developer Productivity & Culture — Thu, 04 Jun 2015 03:00:00 GMT

With all the new technologies emerging for cloud migration, there is a wide acceptance that cloud services are the cheapest, and best way to manage some kinds of infrastructures. Nevertheless, on the decision on wether we should migrate into cloud must still be analysed in other ways. That’s why the current Milk-as-a-Service analogy is great, as it illustrates the downfalls of moving to cloud, and the exact situation on wether you need a migration or not.

Abraham Lincoln: Vampire Hunter

Apurva Shukla's RSS Feed — Wed, 03 Jun 2015 10:00:00 GMT

⭐ ⭐ ⭐ ⭐ It was a great read I must say! Packed with conspiracy and alternate history while remaining true to the facts of Lincoln’s life…

Coding Pet Peeves

The Cranky Developer on Crater Moon Development — Wed, 03 Jun 2015 03:00:00 GMT

Eight little things I like to nitpick about in code. Mostly applicable to Java, but some are language-neutral.

Come In By Robert Frost

ho.dges.online — Wed, 03 Jun 2015 03:00:00 GMT

As I came to the edge of the woods,
Thrush music – hark!
Now if it was dusk outside,
Inside it was dark.

Too dark in the woods for a bird
By sleight of wing
To better its perch for the night,
Though it still could sing.

The last of the light of the sun
That had died in the west
Still lived for one song more
In a thrush’s breast.

Numberphilia

Donat Studios — Tue, 02 Jun 2015 06:13:00 GMT

There's been a fair deal of hubbub in the community about version numbers, including what is and isn't Semantic Versioning, what qualifies as a breaking change and how to use version numbers.

Underscore.js released a second digit "minor update" that was actually a breaking update for some people. They argue it was minor because not very many people would be affected. The community however was angry.

On the opposite side of things, PHP Internals announced they will be skipping version 6 and numbering the next version of PHP 7 to avoid confusion with the never released UTF-16 native version. Again the community was angry.

Using Version Numbers Well

If you are semantically versioning your library, the version represents a contract of trust between you and your developers regarding the public API. The contract declares that nothing you change in a 'less than major' release should break their code. Ever.

Break the contract, you can end up breaking their trust. Break their trust and they'll look elsewhere.

Version Numbers SHOULD NEVER Be Branding

If you're using your version numbers as branding, you are doing it wrong. Version numbers should show a clear history of the public facing interface, not what you sell or advertise it as.

Microsoft of all people understands this. While their version numbers aren't truly semantic, they are aimed at developers - and highly out of sync with the marketing designation because the version numbers denote a true relationship.

You can have both a release number and a sales version. This is the way to go if sales demands control of the version number, such as where I work.

Increment Early, Increment Often

Breaking changes always need to be a first digit release. Always. It doesn't matter if it's not a common use case. It shouldn't matter if you're saving the number for a "woo big new release" version. A breaking change in a minor release violates trust and brings the quality of your software into question.

WHENEVER a behavior or signature of ANY explicitly public member of your API has changed whereas the previous behavior was anything other than bold face wrong it is a first digit release.

Numbers are Cheap

Numbers are cheap. Numbers are infinite. Never reuse numbers, ever. There is no reason.

PHP in my strong opinion did the right thing by skipping 6. They sidestepped ANY potential confusion, minor as it may have been. The argument that they caused confusion about "Where did 6 go" is childish. People will be investing time in the largest digit release, regardless of gaps. No one (save some overly cautious IT people) is going to be using PHP 5.6 and on seeing 7 released think "I'll upgrade to 6".

Trying to not have gaps in your version numbers makes as much sense as if Git were to use ordered numbers instead of hashes.

Version numbers should be a universally unique identifier for a specific set of code. If there is ever any form of contention over what a version number represents, increment. Don't think about it - just do it.

Conclusion

Always avoid confusion; don't ever be afraid to increment your major release number. Only good things can come from it.

Trust is maintained.

Using msmtp to handle mail delivery

Erethon's Corner — Tue, 02 Jun 2015 03:53:22 GMT

Following the example of GKH and others, I've added one more piece of software on my email setup and workflow. That piece is msmtp, a very simple and lightweight SMTP client, that integrates really well with the rest of my setup.

Xcode bookmarks

xenodium.com @alvaro — Tue, 02 Jun 2015 03:00:00 GMT

Costa Rica travel bookmarks

xenodium.com @alvaro — Tue, 02 Jun 2015 03:00:00 GMT

Australia travel bookmarks

xenodium.com @alvaro — Tue, 02 Jun 2015 03:00:00 GMT

Australia's best food experiences: state by state.
Best Queensland island escapes for small budgets.
Des and Debi O’Tooles Honey.
Fish and chips at Bondi beach.
Tasmania: the isle that's wild at heart.
The Butler Potts Point (bar & restaurant).
Tree Top Walk (Walpole, Australia): Top Tips Before You Go - TripAdvisor.

Samoa travel bookmarks

xenodium.com @alvaro — Tue, 02 Jun 2015 03:00:00 GMT

To Sua ocean trench.

Norway travel bookmarks

xenodium.com @alvaro — Tue, 02 Jun 2015 03:00:00 GMT

Atlantic Road.
Bergen (check out colorful wooden houses).
Bergen railway (Bergen-Oslo): 300 miles of beautiful Norwegian scenery.
Lofoten Islands.
Norwegian Air (cheap flights between all the
Olden.
Reine.
Sakrisøy, Lofoten Islands.
Spitsbergen.
The Atlantic Road - Atlanterhavsveien / Atlanterhavsvegen.

Los Angeles travel bookmarks

xenodium.com @alvaro — Tue, 02 Jun 2015 03:00:00 GMT

Rails: Writing Engine Tests That Depend on Main Application Models

kyrofa's blog — Tue, 02 Jun 2015 03:00:00 GMT

When developing a new Ruby on Rails project, one doesn’t always start from scratch. Often one first searches for an open-source project that will at least get one part of the way there. For example, one might want to do something as simple a create a website with a blog as well as image galleries. The first step in this journey might be to find some open-source blog software for Rails, e.

The right tool for the job: Choosing where to use RSVP-TE or SR.

rob.sh — Tue, 02 Jun 2015 01:45:00 GMT

I noted that at NANOG64 this week in San Francisco, there are talks (both from Juniper) about both SPRING/Segment Routing and RSVP-TE. These are both protocols/technology approaches (since one can’t really call SR a protocol) that I’ve been involved in the evolution of over the last few years. A question that I’ve been asked more times than I’d like is why we chose to look at a new approach (SR) rather than go with a technology that exists RSVP-TE.

Some Thoughts on Giving a Good Talk About Your Research

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Sun, 31 May 2015 18:42:17 GMT

I just had the privilege of watching lots of great talks at the EuroVis 2015 conference. Since giving a talk about your research is a common occurrence, even for undergraduate students, I thought it worthwhile to think a little bit about what makes a talk truly great.

It goes without saying these are my personal opinions. They reflect the ways in which I perceive a talk. Still, here are some rules that good talks tend to adhere to:

Preparing a good talk takes time. You should figure that into your calculations. To prepare a good talk, several iterations are probably required. Test it on your friends, your colleagues, your family, and so on.
Avoid last-minute changes. Been there, done that. Try to avoid it unless your colleagues or supervisor have some specific criticisms about your slides that can be fixed. Do not do it on your own.
Know your audience. Your audience determines the level of specificity your talk should have. An all-expert audience can take more technical details than a more general audience. If in doubt, follow the rule of thirds: Prepare one third of your talk for the experts, one third for your peers, and one third for the general listener.
Avoid death-by-detail. This is somewhat interlinked with the previous rule. A good research talk should show the interesting challenges in your research. It should make people want to read your thesis, your paper, or discuss stuff with you. The quickest way to lose an audience is to bog them down with too many details. When in doubt, always opt for the bird’s eye view. Technical details can always be read in the paper or discussed face-to-face. Of course, you should outline the challenges you had to face and the obstacles your research overcame. If this involves some details, so be it. Just make sure that people who have not (yet) read your paper will follow you.
Know the format of your talk and know what it should accomplish. Lest you think that I advocate a marketing presentation in the previous rule, rest assured. If you have one hour of speaking time to convince a funding committee that your work is great or your advisers to give you the coveted degree, it is probably a good idea to include as many details as possible. At a conference, however, you have to convince people to read your paper. These two goals have a very different nature and need to be tackled differently.
Whenever possible, give the audience time to breathe again. So you just waltzed through a very difficult mathematical technique? Add a slide in which you explain why this was done. Give the audience a chance to catch their breath. Remind them of the overall goal you want to accomplish, for example showing that a certain algorithm scales to larger data sets. This gives the audience the possibility to nod their heads in agreement and see your motivation, even if they did not understand everything perfectly.
Consistency. Try to be as consistent as possible, with respect to colours, names, and even the layout of your slides. Is it possible to refer to your method as either foo or bar? Pick only one. Are there two possibilities A and B for solving something? Show things pertaining to A always in the same colour. Use a different colour for B.
Add visual metaphors. Does your method work like a cheese grater? Or like a meat grinder? Use these metaphors to get members of the audience to understand what you are talking about. However, do not attempt to force a metaphor if there is none. I would rather have you talk about the differential structure of non-Euclidean manifolds than try to convince me that they “work exactly like a trash can…”
Make a test-run of your presentation. Bring all your adapters. Make yourself familiar with how to connect your notebook to the projector. Check that your operating system is using the proper resolution, and so on. There is nothing worse than being nervous and having to fiddle for what feels like an eternity with different cables, settings, and so on. If your talk is virtual, familiarise yourself with the conferencing software.
Speak freely. If you are very inexperienced with giving a talk, reading off a fully-formulated talk might put you at ease. It is however also possible that you stumble over one sentence and are completely thrown off course. I saw this happen multiple times and it is very hard to recover from this. I’d rather have a presenter speaking freely and sounding authentic than a completely-polished talk that is only read-off.

These are some of the rules I would like to impart on people who have to give a talk. Finally, the most important rule of all: Have fun and do not forget that other people are nervous as well. The final rule does not get a number because 10 rules are much better than 11.

Enjoy your talks, be it as a speaker or an audien ce member.

Simple fractals with Qt 5 shaders

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Sun, 31 May 2015 18:04:52 GMT

I wanted to play around with some shaders recently so I was looking for simple things to implement. One of the nicest ways to get started in a new graphics framework is to get some fractals going—so this is what I did.

The Mandelbrot shader

We first require some GLSL magic to make the Mandelbrot fractal happen:

uniform int iterations;

uniform highp vec2 centre;
uniform highp float scale;

varying highp vec2 texture_out;

void main()
{
  vec2 z;
  vec2 c;
  c.x = scale * ( 3.0 * texture_out.x - 2.0 ) + centre.x;
  c.y = scale * ( 2.0 * texture_out.y - 1.0 ) + centre.y;

  z = c;

  int i = 0;
  for(; i < iterations; ++i)
  {
    float x = z.x*z.x - z.y*z.y + c.x;
    float y = z.x*z.y + z.y*z.x + c.y;

    if( x*x + y*y > 4.0 )
      break;

    z.x = x;
    z.y = y;
  }

  vec4 color = vec4(0.0);

  if(i < iterations - 1)
  {
    color.x = sin(float(i) / 3.0);
    color.y = sin(float(i) / 6.0);
    color.z = cos(float(i) / 12.0 + 3.141 / 4.0);
  }

  gl_FragColor = color;
}

Basically, I am filling the texture coordinates with colours, based on the number of iterations it took for the series to reach the desired norm of 4.0. The scale and centre parameter exist only for the purpose of modifying the view within the controlling application.

The Julia set shader

Now let’s add another shader for the Julia set:

uniform int iterations;

uniform highp vec2 c;
uniform highp vec2 centre;
uniform highp float scale;

varying highp vec2 texture_out;

void main()
{
  vec2 z;
  z.x = scale * ( 3.0 * texture_out.x - 2.0 ) + centre.x;
  z.y = scale * ( 2.0 * texture_out.y - 1.0 ) + centre.y;

  int i = 0;
  for(; i < iterations; ++i)
  {
    float x = z.x*z.x - z.y*z.y + c.x; 
    float y = z.x*z.y + z.y*z.x + c.y; 

    if( x*x + y*y > 4.0 )
      break;

    z.x = x;
    z.y = y;
  }

  vec4 color = vec4(0.0);

  if(i < iterations - 1)
  {
    color.x = sin(float(i) / 3.0);
    color.y = sin(float(i) / 6.0);
    color.z = cos(float(i) / 12.0 + 3.141 / 4.0);
  }

  gl_FragColor = color;
}

Note that the differences between the shaders are minuscule at best.

Using Qt 5 to put it together

Putting the fractals on the screen then turned out to be a rather easy exercise with Qt 5. Using a QGLShaderProgram, I only need to render a quad onto the screen, bind the appropriate shader, and we are done. Check out the code on GitHub for more details.

Screenshots

Click to see the screenshots in their original resolution.

The Mandelbrot fractal in all its glory. Such colours, such self-similarity. Wow!

A zoomed-in portion of the Mandelbrot fractal. Still nice, I guess.

A particular view on the Julia set.

Why Other Mapping Platforms Don't Cut It Anymore

unsungNovelty — Sun, 31 May 2015 09:34:34 GMT

Note: This post was written in 2015. The writing quality is not that great and I won’t try to improve it as well. This was republished from another platform for pure historic value of being my first blog post ever. My opinions and thoughts on the topics shared in this post might have changed.

A Recap on the discovery of Openstreetmap and the two days in Mapbox.

Maps were something that I was really good at! Or something I didn’t know much about until I bumped into Openstreetmap (OSM). I thought I was an expert in maps as I was using Google maps application. Keep it updated to the latest version, and i was doing only that. Come on.. I was working in Bangalore - A place I knew nothing about initially. I traveled less but when I did, I was using Google maps to tell me how to go to places. I didn’t know from where these guys where getting the data about everything. I mean, who cares right? I just wanted to go to those places. I let Google do all the work for me, And I was fine with it too! That’s when I got to know about Openstreetmap - which is an open source project.

Openstreetmap - It was nothing like the other open source projects. You contribute map data instead of the codes (Most of the part I guess). Wait… what? Map data? How do I get it? From where do I get it? Should I be copying from Google Maps…? I had no idea! My mind was full of questions. As days passed, I was learning a lot about maps. Got to know a lot on what kind of data will be uploaded to openstreetmap, and the details that completes a map. I didn’t do much after that though. It was just another awesome project that I bumped into like many other.

I applied for the job, got acknowledged by Sajjad. We talked a lot about what they were working on in Mapbox. That’s when i got to know about the connection of Mapbox and Openstreetmap. I wanted to know more about Openstreetmap!

Luckily my friend who had a good knowledge on OSM helped me out with it. And in the process I got to know about JOSM (A cross-platform editor for openstreetmap written in Java), the most used tool by OSM contributors!

As planned earlier, I had the call with Sajjad where we talked about what my goals and passions were and reason for applying this job. I liked where the discussion was heading and how the company and I shared the same interests (Open source!!). After that I started mapping in a pretty small way…. Started using OSM tracker for Android - which is an app on Google play store to track the GPS co-ordinates of your journey using your smartphone GPS. Went to some of the places in my village and got the raw data as .gpx files. Started adding the data and uploaded the same to the Openstreetmap.org. My first meaningful contribution to an open source project!

Discussions were going on in mails and I was contacted by Arun from Mapbox and he gave me a much needed idea on what my role will be if i were to work full time on Mapbox. I learned a lot through the discussions and Arun was always there to help me with the answers for my queries. That was when Earthquake happened in Nepal and later in some parts of India. That is also the time when I knew the value of crowd sourcing and power of the Openstreetmap community. They updated the areas affected by earthquakes in two weeks or so. I was amazed by the sheer power Openstreetmap community. The changes made by them was significant compared to before and after the disastrous earthquake. This was amazing according to me because I have never seen anything like that till then. I was busy with my work that I have in Tech Mahindra (TechM) at this time. And because of this, I couldn’t be a contributor among the thousands who helped update the maps in Nepal post earthquake. Later I started working on Agra road network and completed a task and uploaded the same to OSM. It was a wonderful feeling and experience.

Conversations were happening with me and Arun through mails quite often and I was getting closer and closer to OSM and Mapbox. It was at this time a call was arranged with Alex Barth who leads the data team in Mapbox. We had a discussion about what I did in TechM (the company that I work in currently), and how the clients of Mapbox uses the maps. Everything was interesting and I got to know much more about the company. After some days, I was called to Mapbox office for two days sprint.

The meetup

Day-1

It was only 30mins travel from my room in Bangalore, so that was not a big issue for me. Reached Mapbox office at 8.35am or so. Nobody except Shravan was in the office. Met him and was on to work right away. We started tracing out buildings in Gangtok and after a while came Arun. Finally got to meet him. After everyone was in the office, we started to plan to go out for surveying parts of Indiranagar. For this we used Fieldpapers which is a website that helps us with Hand-drawn graphic to represent making a Field Papers atlas. It’s like, we will be taking the print out of an area, taking the survey and mark the changes in the paper. After the changes are marked, we can scan the sheet and upload the picture file to https://www.fieldpapers.org and get it aligned properly with OSM maps in JOSM. The area to be covered was split into two namely A1 and A2. A1 containing 4 people and A2 containing the other 4 people. That was further split into two. Two for covering the Point of Interests (POI’s) and two people covering the trees and levels of the buildings. This is easier said than done. We had to go to each streets and crosses in the area to get all of this data and Me and Ruth from Mapbox was one team. Our job was to get the Point of Interests (POI’s) of the areas in A2.

After the data was collected, which was a tiresome walk in the sun but fun at the same time we came back to the office so that we can start uploading the changes to OSM. As the fieldpapers.org aligning mechanism was not working as it’s suppose to be, we had this fun time learning to do the same in JOSM manually. Once the page was aligned exactly on top the OSM maps, we started uploading the changes to OSM. After a long period of continuous effort we all were able to upload the changes to OSM which had the names of all the shops and levels of the buildings and the trees that was there in the area. Yes.. trees!!! We had an awesome time after the work with the team talking and playing Jenga. Sajjad was playing this awesome song through his phone which i got to know was || Kygo – Firestone || which became one of my favorite song soon after.

Day-2

The second day started with Arun teaching what all is that we do in Mapbox as a Data analyst (which is the designation that I applied for). Learned a lot about how to manage the different kind of issues that come in daily. After that, Arun was showing the map that he have created about the Lakes in the south Bangalore with the help of OSM data. I was surprised and amazed to know that all the lakes were inter-connected! All the data that was in the maps was showing how the Bellandur lake was the meeting point where all the polluted and dirty water was flowing in. This partially explained why the lake is so polluted and how the fire emerged from the lake and the reason for the foam that was seen above the lake water like a layer. Everything proving the real power of crowd sourcing and opens source once again. After learning how to handle various types of issue that we may face on a daily basis, I knew it was time to call it a day. The two days went just like that with lots of fun and lots of new things learned. Thanks to Mapbox, I learned a lot on how you can trust to use an open source project as a base layer upon which a business can be made with the help of crowd sourcing and open source community it will offer.

Have thoughts or feedback regarding my post/writing? DM me on Twitter @unsungNovelty.

How to find a commit that introduced a specific change with Git Bisect

Posts by Petr Stribny — Sat, 30 May 2015 15:27:16 GMT

Complex Room Renders

Code & Visuals — Sat, 30 May 2015 03:00:00 GMT

I realize I have not posted in some weeks now, which means I still haven’t gotten around to writing up Takua a0.5’s architecture and VCM integrator. I’m hoping to get to that once I’m finished with my thesis work. In the meantime, here are some more pretty pictures rendered using Takua a0.5.

A few months back, I made a high-complexity scene designed to test Takua a0.5’s capability for handling “real-world” workloads. The scene was also designed to have an extremely difficult illumination setup. The scene is an indoor room that is lit primarily from outside through glass windows. Yes, the windows are actually modeled as geometry with a glass BSDF! This means everything seen in these renders is being lit primarily through caustics! Of course, no real production scene would be set up in this manner, but I chose this difficult setup specifically to test the VCM integrator. There is a secondary source of light from a metal cylindrical lamp, but this light source is also difficult since the actual light is emitted from a sphere light inside of a reflective metal cylinder that blocks primary visibility from most angles.

The flowers and glass vase are the same ones from my earlier Flower Vase Renders post. The original flowers and vase are by Andrei Mikhalenko, with custom textures of my own. The amazing, colorful Takua poster on the back wall is by my good friend Alice Yang. The two main furniture pieces are by ODESD2, and the Braun SK4 record player model is by one of my favorite archviz artists, Bertrand Benoit. The teapot is, of course, the famous Utah teapot. All textures, shading, and other models are my own.

As usual, all depth of field is completely in-camera and in-renderer. Also, all BSDFs in this scene are fairly complex; there is not a single simple diffuse surface anywhere in the scene! Instancing is used very heavily; the wicker baskets, notebooks, textbooks, chess pieces, teacups, and tea dishes are all instanced from single pieces of geometry. The floorboards are individually modeled but not instanced, since they all vary in length and slightly in width.

A few more pretty renders, all rendered in Takua a0.5 using VCM:

"Leaving out the best bit" project management anti-pattern

Databases & Life — Sat, 30 May 2015 03:00:00 GMT

When I was at university in the 90s, studying computer science, my final-year bachelor project was a “fractal raytracer”.

My idea was to write a program which could produce pictures of those wonderful artificial worlds one saw on posters of the time, hung in every university dorm room. Raytracers render highly realistic images, and fractals produce interesting shapes to render.

I set about designing the program, with all the features I wanted. In the object-oriented language of the day (Objective-C) I envisaged a Shape class with various subclasses to represent the various objects the program should be able to render: spheres, fractal landscapes, etc. The fractal landscapes would be further subdivided into those more random mountainous landscapes and, the crowning glory, a 3-dimensional landscape based on those particularly recognizable fractal shapes such as the Mandelbrot.

Blade Runner

🦉 olshansky 🦁 — Fri, 29 May 2015 06:00:22 GMT

Olshansky's review of Blade Runner

You Don't Need jQuery!

Train of Thought — Fri, 29 May 2015 03:00:00 GMT

A seven-part series that will help free you from the chains of jQuery by showing you how to embrace and understand the modern web API and JavaScript.

Mastering Emacs is out

xenodium.com @alvaro — Thu, 28 May 2015 03:00:00 GMT

Emacs is amazingly alive. New packages are regularly listed on melpa and a new book just came out: Mastering Emacs by Mickey Petersen.

Important Factors To Evolution

Venam's Blog — Patrick Louis (Lebanon) — Thu, 28 May 2015 00:00:00 GMT

Hello fellow readers, The last two months I've been extremely interested in evolutionary programming and group dynamics. In this post I'll try to gather some important factors I've noticed repeating themselves across the multiple opinions and examples, and the ones that just seemed noticeably important.

12-Factor Apps in node

Pete's Points — Wed, 27 May 2015 17:33:52 GMT

Here's my talk for the node.js Boulder meetup group about 12-Factor apps in node.js. Slides are here

Items

Liza Shulyayeva — Wed, 27 May 2015 14:58:05 GMT

After finishing yesterday’s post about generating images for user-combined substrate items I realized that I never really provided an explanation about how items work in general (or how they work for now, anyway).

Escaping Daddy

Apurva Shukla's RSS Feed — Wed, 27 May 2015 11:06:22 GMT

⭐ ⭐ ⭐ 3.5/5 As a reader I scoffed at moments that Maria made decisions that were obviously for the detriment of herself, but having realised…

Buck - life is too short to spend a minute for each build

zserge's blog — Wed, 27 May 2015 03:00:00 GMT

Gradle is too slow, Ant is too basic. Buck is fast and super-configurable and I’d like to tell you how I use it for Android development. I own a pretty old thinkpad laptop so I don’t expect the builds to be too fast. Ant used to spend 10..40 seconds for each build on my laptop depending on how large the project is. Gradle takes about a minute in average and that’s too long for me - I just lose focus when I just sit and wait for the build results.

Images for combined substrate

Liza Shulyayeva — Tue, 26 May 2015 23:31:57 GMT

The Laravel 5 migration is complete and I’m back to substrate mixing and brain bug fixing!

Actually, I think now that basic mixing is done I’m going to do a few weeks of just bug fixing. The substrate has no effect on snails’ attributes or behaviour yet, but it will. For now you can just mix different substrate items together into new substrate items.

Putting Test Case Specification directly in code

Posts by Petr Stribny — Tue, 26 May 2015 18:16:21 GMT

Canadian Real Estate or American Stock?

🦉 olshansky 🦁 — Tue, 26 May 2015 11:49:03 GMT

tl;dr Investing in the S&P 500 is more profitable than buying a condo in Toronto with a 25% downpayment and renting it out while still…

Composition vs. Inheritance: How to Choose?

Nicola Iarocci — Tue, 26 May 2015 03:00:00 GMT

Steven Lowe piece on Composition vs. Inheritance is a must read. Just to wet your appetite, let me quote the opening paragraph:

In the beginning, there was no inheritance and no composition, only code. And the code was unwieldy, repetitive, blocky, unhappy, verbose, and tired. Copy and Paste were the primary mechanisms of code reuse. Procedures and functions were rare, newfangled gadgets viewed with suspicion. Calling a procedure was expensive! Separating pieces of code from the main logic caused confusion! It was a Dark Time.

Mathematics The Law Of the World

Venam's Blog — Patrick Louis (Lebanon) — Tue, 26 May 2015 00:00:00 GMT

Hello fellow readers, This post is about a little adventure I had with maths. I'm currently reading a book called `clever algorithms`, I go along at my own pace doing researches on hard subjects or things I forgot from high school. Digging through the cross entropy algorithm I had to take a break and learn more about distribution. Namely, I wanted to refresh my mind about the normal/Gauss law.

South Carolina travel bookmarks

xenodium.com @alvaro — Mon, 25 May 2015 03:00:00 GMT

Angel Oak tree.

Colorado travel bookmarks

xenodium.com @alvaro — Mon, 25 May 2015 03:00:00 GMT

Horseshoe Bend.

Cuzco travel bookmarks

xenodium.com @alvaro — Mon, 25 May 2015 03:00:00 GMT

Gelatina de patita (mercado)
Yolanda. Adobo/chicarron. Chicarroneria Yoli.
Pan de chocolate Oropeza.
La esquina de los lechones y tamales dulces de Sra Elsa.
Cafe el Ayllu (lengua de suegra)
La Chomba (picanteria). Costillar frito. Frutillada. Chicharron. Ubre. Tripa.
La quinta Eulaia. Asado y chicarron. Rocoto. Humita.
Chicheria la loba.

Bash bookmarks

xenodium.com @alvaro — Mon, 25 May 2015 03:00:00 GMT

restclient.el

xenodium.com @alvaro — Mon, 25 May 2015 03:00:00 GMT

Installed Pashky's restclient.el Emacs package. Super helpful when trying out REST APIs.

Seatle travel bookmarks

xenodium.com @alvaro — Mon, 25 May 2015 03:00:00 GMT

Berlin travel bookmarks

xenodium.com @alvaro — Mon, 25 May 2015 03:00:00 GMT

Skeuomorph

xenodium.com @alvaro — Mon, 25 May 2015 03:00:00 GMT

From Wikipedia, skeuomorph ˈskjuːəmɔrf is a derivative object that retains ornamental design cues from structures that were necessary in the original. Examples include pottery embellished with imitation rivets reminiscent of similar pots made of metal and a software calendar that imitates the appearance of binding on a paper desk calendar.

define-word

xenodium.com @alvaro — Mon, 25 May 2015 03:00:00 GMT

Installed Abo Abo's define-word Emacs package. A handy package to define words at point.

The "New" New Microsoft

Dan Stroot — Mon, 25 May 2015 03:00:00 GMT

I've been a fan of Microsoft during my whole technical career. However I don't recommend Microsoft any longer for server workloads, and haven't for years due to the fact that the Linux ecosystem is far less expensive and has more innovative people working on it and sharing what they do openly. However, I am very impressed with the direction and leadership of Satya Nadella.

This post The "New" New Microsoft first appeared on Dan Stroot's Blog

HtmlToWord is now WordInserter

Tom Forbes — Sun, 24 May 2015 23:57:06 GMT

I’ve released a redesign of my HtmlToWord library, specifically it now supports Markdown and multiple different ways to interact with Word. It’s now also been renamed to WordInserter to reflect this. Originally HtmlToWord was designed to take HTML input, process it and then insert a representation o...

Unity 8's Snappy Scope

kyrofa's blog — Sun, 24 May 2015 03:00:00 GMT

I recently began a new position at Canonical. I joined this company in particular to give myself more opportunities for growth. Now, when I say that, people tend to assume I’m talking about a managerial track. I’m not sure why they tend to jump there, but it’s not what I’m referring to here. I’m talking about technical growth. I feel at the top of my game when I’m learning something brand new, and I needed a company that facilitated that.

Note On Images

Code & Visuals — Sat, 23 May 2015 03:00:00 GMT

Just a quick note on images on this blog. So far, I’ve generally been embedding full resolution, losslessly compressed PNG format images in the blog. I prefer having the full resolution, lossless images available on the blog since they are the exact output from my renderer. However, full resolution lossless PNGs can get fairly large (several MB for a single 1920x1080 frame), which is dragging down the load times for the blog.

Going forward, I’ll be embedding lossy compressed JPG images in blog posts, but the JPGs will link through to the full resolution, lossless PNG originals. Fortunately, high quality JPG compression is quite good these days at fitting an image with nearly imperceptible compression differences into a much smaller footprint. I’ll also be going back and applying this scheme to old posts too at some point.

Addendum 2016-04-08

Now that I am doing some renders in 4K resolution (3840x2160), it’s time for an addendum to this policy. I won’t be uploading full resolution lossless PNGs for 4K images, due to the overwhelming file size (>30MB for a single image, which means a post with just a handful of 4K images can easily add up to hundreds of MB). Instead, for 4K renders, I will embed a downsampled 1080P JPG image in the post, and link through to a 4K JPG compressed to balance image quality and file size.

2015-05-23

ho.dges.online — Sat, 23 May 2015 03:00:00 GMT

Flushing empty lines in Emacs

xenodium.com @alvaro — Fri, 22 May 2015 03:00:00 GMT

Via masteringemacs.org, removing blank lines in a buffer:

M-x flush-lines RET ^$ RET

Regex bookmarks

xenodium.com @alvaro — Fri, 22 May 2015 03:00:00 GMT

Spring Cleaning

Sam Solomon — Fri, 22 May 2015 03:00:00 GMT

I recently left my job in Advertising to move back to Atlanta and build a fantasy sports company. The guy I was going to room with had a month left on his lease.

Blindness by Iain Banks

ho.dges.online — Fri, 22 May 2015 03:00:00 GMT

Fattura Elettronica Open Source: Web Service PA

Nicola Iarocci — Thu, 21 May 2015 03:00:00 GMT

this post is about an all-Italian open source release, so it’s going to be in Italian

Il progetto Fattura Elettronica Open Source si è arricchito di un nuovo strumento: Web Services. Il namespace FatturaElettronicaPA.WebServices raccoglie una serie di client C# che consentono di consultare i Web Service per la Fattura Elettronica messi a disposizione dalla Pubblica Amministrazione.

Sono disegnati in maniera da esporre tutti la stessa interfaccia ed essere al tempo stesso semplici e leggeri. Al momento lavorano in modalità sincrona ma l’obiettivo è di renderli tutti asincroni.

The Five Rules for Successful Stock Investing: Morningstar's Guide to Building Wealth and Winning in the Market

🦉 olshansky 🦁 — Thu, 21 May 2015 03:00:00 GMT

Olshansky's review of The Five Rules for Successful Stock Investing: Morningstar's Guide to Building Wealth and Winning in the Market by Pat Dorsey

Be Careful When The Offer You The Moon

ho.dges.online — Mon, 18 May 2015 03:00:00 GMT

Be careful when they offer you the moon
It gives a cold light
It was only ever made to light the night
You can freeze your fingers handling the moon

Be careful when they offer you the moon
It’s built for dead souls
It’s a colourless and dusty ball of holes
You can break and ankle dancing on the moon

When you take the moon you kiss the world goodbye
For a chance to lord it over loneliness
And a quarter-million miles down the sky
They watch you shining more but weighing less

Subclassing UILabel in 2015

Whacky Labs — Sat, 16 May 2015 09:23:54 GMT

UILabel is one the most smart view in UIKit family. It knows a lot about itself. If you constraint it to a certain width, the UILabel can calculate the height for itself. Another smart thing about UILabel is that is saves you machine cycles by not redrawing its content unless something is actually modified. And if you use NSAttributedString, you can in fact draw a more sophisticated text content.

Given so many good things with a UILabel, it is very tempting to subclass a UILabel to render our custom text content. Say if you’re working on a game or app, say a chat messenger, you may like to have a view where you render you text and have some decorations around it, like a speech bubble with an optional image. And, if you’ve ever worked on such a task, you know how painful it actually is!.

A UILabel is the perfect fit whenever you’ve a need to render some text on screen. Recently had to undergo such a job of subclassing UILabel, and it surprisingly hard to find a good source of information that shows how to properly subclass UILabel. It might work for the text data you have right now, but there are so many screen sizes and edge cases possible. If you’re one of those person, I don’t want you to discover this knowledge the hard way like I did, I’ll share the knowledge I’ve acquired from my adventure.

Even if you’re not directly using UILabel but rendering your content on an OpenGL canvas using Freetype library or something, the knowledge of how UILabel works should be really helpful in designing your own view. But, you can take my word for it, that rendering text is one of the most challenging things in the UI problem domains. You have to render all sort of glyphs, in all sort of writing directions with a lot of kerning, ligatures and other text attributes. If you’re one of those adventure seeker, you should definitely try building your own text rendering system. Just for the inspiration, these are from the days when I was building one:

Fixing font rendering the long way pic.twitter.com/hJtCBKLqBN
— Sidharth Juyal (@chunkyguy) June 26, 2013

One day work = FreeType font rendering pic.twitter.com/FJMrcPPu
— Sidharth Juyal (@chunkyguy) February 6, 2013

Someday, I should probably also share my experiences with building a UILabel-like view from scratch, I mean on top of freetype. But, today we just extend over the work UILabel already does for us. Let’s begin with creating a new project.

We simply create a new project with a single view and add a RoundedRectLabel class. Next, we write our basic code to render the label on screen.

class RoundedRectLabel: UILabel {
}

class ViewController: UIViewController {
    override func viewDidLoad() {
        super.viewDidLoad()
        view.backgroundColor = .lightGrayColor()
        let chatBubbleLabel = RoundedRectLabel()
        chatBubbleLabel.text = "No one expects the Spanish Inquisition! Our chief weapon is surprise. Fear and surprise. Two chief weapons, fear, surprise, and ruthless efficiency! Er, among our chief weapons are: fear, surprise, ruthless efficiency, and near fanatical devotion to the Pope! Um, I'll come in again."
        chatBubbleLabel.textColor = .blackColor()
        chatBubbleLabel.backgroundColor = .cyanColor()
        chatBubbleLabel.numberOfLines = 0
        chatBubbleLabel.textAlignment = .Justified
        view.addSubview(chatBubbleLabel)
        chatBubbleLabel.setTranslatesAutoresizingMaskIntoConstraints(false)
        view.addConstraint(NSLayoutConstraint(item: chatBubbleLabel, attribute: .CenterY, relatedBy: .Equal, toItem: view, attribute: .CenterY, multiplier: 1.0, constant: 0.0))
        view.addConstraints(NSLayoutConstraint.constraintsWithVisualFormat("H:|-[chatBubbleLabel]-|", options: NSLayoutFormatOptions(0), metrics: nil, views: ["chatBubbleLabel": chatBubbleLabel]))
    }

    override func prefersStatusBarHidden() -> Bool {
        return true
    }
}

Now the two important methods to subclass within a UILabel are:

func textRectForBounds(bounds: CGRect, limitedToNumberOfLines numberOfLines: Int) -> CGRect
func drawTextInRect(rect: CGRect)

The textRectForBounds(_:limitedToNumberOfLines:) provides us an opportunity to update the drawing area of the text within the UILabel. And the drawTextInRect() is for you to do you custom drawing within the view.

If you notice, this is a little different than your usual UIView subclassing, where you typically draw you content in drawRect(). Another thing to keep in mind is that you typically also don’t want to modify the intrinsicContentSize(). Overriding these two methods will do whatever you want to do with you custom UILabel.

For sake of getting a deeper understanding, let’s print out the order of execution.

class RoundedRectLabel: UILabel {
   override func textRectForBounds(bounds: CGRect, limitedToNumberOfLines numberOfLines: Int) -> CGRect {
        let textRect = super.textRectForBounds(bounds, limitedToNumberOfLines: numberOfLines)
        println("\(__FUNCTION__): \(textRect)")
        return textRect
    }

    override func drawTextInRect(rect: CGRect) {
        super.drawTextInRect(rect)
        println("\(__FUNCTION__): \(rect)")
    }

    override func intrinsicContentSize() -> CGSize {
        let size = super.intrinsicContentSize()
        println("\(__FUNCTION__): \(size)")
        return size
    }
}

Here’s the output:

textRectForBounds(_:limitedToNumberOfLines:): (0.0, 0.0, 2122.5, 20.5)
intrinsicContentSize(): (2122.5, 20.5)
textRectForBounds(_:limitedToNumberOfLines:): (0.0, 0.0, 342.5, 142.0)
intrinsicContentSize(): (342.5, 142.0)
drawTextInRect: (0.0, 0.0, 343.0, 142.0)

The interesting thing to notice here is that whatever is calculated from the super.textRectForBounds(_:limitedToNumberOfLines:) is also passed on to the intrinsicContentSize(). Another interesting thing is that, the methods are called multiple times. First time with some estimated values by UIKit and second time after the values have been evaluated accurately enough.

Not implementing the intrinsicContentSize() also means that, you don’t need to invalidateIntrinsicContentSize() yourself. Since, the default drawing mode of UILabel is UIViewContentMode.Redraw, the content is redrawn automatically whenever the content updates. To confirm, if you update the text after a while, you can see the drawing methods being invoked again.

class ViewController: UIViewController {
    override func viewDidLoad() {
        super.viewDidLoad()
        view.backgroundColor = .lightGrayColor()
        let chatBubbleLabel = RoundedRectLabel()
        chatBubbleLabel.text = "No one expects the Spanish Inquisition! Our chief weapon is surprise. Fear and surprise. Two chief weapons, fear, surprise, and ruthless efficiency! Er, among our chief weapons are: fear, surprise, ruthless efficiency, and near fanatical devotion to the Pope! Um, I'll come in again."
        chatBubbleLabel.textColor = .blackColor()
        chatBubbleLabel.backgroundColor = .cyanColor()
        chatBubbleLabel.numberOfLines = 0
        chatBubbleLabel.textAlignment = .Justified
        view.addSubview(chatBubbleLabel)

        chatBubbleLabel.setTranslatesAutoresizingMaskIntoConstraints(false)
        view.addConstraint(NSLayoutConstraint(item: chatBubbleLabel, attribute: .CenterY, relatedBy: .Equal, toItem: view, attribute: .CenterY, multiplier: 1.0, constant: 0.0))
        view.addConstraints(NSLayoutConstraint.constraintsWithVisualFormat("H:|-[chatBubbleLabel]-|", options: NSLayoutFormatOptions(0), metrics: nil, views: ["chatBubbleLabel": chatBubbleLabel]))

        dispatch_after(dispatch_time(DISPATCH_TIME_NOW, Int64(NSEC_PER_SEC * 3)), dispatch_get_main_queue()) { () -> Void in
            println("---------")
            chatBubbleLabel.text = "All right, but apart from the sanitation, the medicine, education, wine, public order, irrigation, roads, the fresh-water system, and public health, what have the Romans ever done for us?"
        }
    }

    override func prefersStatusBarHidden() -> Bool {
        return true
    }
}

Output:

textRectForBounds(_:limitedToNumberOfLines:): (0.0, 0.0, 2122.5, 20.5)
intrinsicContentSize(): (2122.5, 20.5)
textRectForBounds(_:limitedToNumberOfLines:): (0.0, 0.0, 342.5, 142.0)
intrinsicContentSize(): (342.5, 142.0)
drawTextInRect: (0.0, 0.0, 343.0, 142.0)
---------
textRectForBounds(_:limitedToNumberOfLines:): (0.0, 0.0, 1403.0, 20.5)
intrinsicContentSize(): (1403.0, 20.5)
textRectForBounds(_:limitedToNumberOfLines:): (0.0, 0.0, 323.5, 101.5)
intrinsicContentSize(): (323.5, 101.5)
drawTextInRect: (0.0, 0.0, 343.0, 101.5)

This is good, because it means if you’re subclassing a UILabel, you need to focus on just these two methods and the UILabel will take care of the rest. So, lets focus first on the drawTextInRect().

The first thing is to calculate the edge insets or padding you wish to give your label.

let edgeInsets = UIEdgeInsets(top: 10, left: 20, bottom: 10, right: 20)

override func drawTextInRect(rect: CGRect) {
    let textRect = UIEdgeInsetsInsetRect(rect, edgeInsets)
    super.drawTextInRect(textRect)
}

As you can see, we’ve the required padding from all edges as we wanted, but the text is truncated at the end. This is because the UILabel currently is trying to draw the text within the original frame it had calculated. So the next task is to provide the new drawing frame to UILabel. This is done within textRectForBounds(_:limitedToNumberOfLines:). The important question is how do we calculate what frame size do we need to draw the entire text content?

The NSString has a handy extension that does this calculations. The method we’re particularly interested in is

func boundingRectWithSize(size: CGSize, options: NSStringDrawingOptions, attributes: [NSObject : AnyObject]!, context: NSStringDrawingContext!) -> CGRect

We just need to provide this method our estimated size of drawing area and other rendering attributes, and it will return the actual frame we need to render this text. This brings us to the part II of the problem: How do we calculate the estimated size of the drawing area?

Let’s see, we know the width that we wish to draw in.

let estimatedWidth = CGRectGetWidth(textRect) - (edgeInsets.left + edgeInsets.right)

But, UILabel passes the same original width to textRectForBounds(_:limitedToNumberOfLines:) and drawTextInRect(). So, this means if the original width of the UILabel is 100 pts and we wish to have a padding of 10 pts from all edges. If NSString API calculates the required height for width = 80, is lets say it’s 200. The size passed down to drawTextInRect() is 100×200, where we again shrink the size down to 80×180. In order to compensate for this second clipping, we must calculate the height for an 2 times smaller width.

let estimatedWidth = CGRectGetWidth(rect) - (2 * (edgeInsets.left + edgeInsets.right))

But what about the height? Don’t worry, this is just an estimated height, we can provide a very high value, and hope for the best.

let estimatedWidth = CGRectGetWidth(rect) - (2 * (edgeInsets.left + edgeInsets.right))
let estimatedHeight = CGFloat.max
let calculatedFrame = NSString(string: text).boundingRectWithSize(CGSize(width: estimatedWidth, height: estimatedHeight), options: .UsesLineFragmentOrigin, attributes: [NSFontAttributeName: font], context: nil)

Next, remember, the boundingRectWithSize(...) will try to wrap our text in as less space as possible, because that is the default behavior. So, we need to explicitly provide the extra top and bottom padding to the size calculated.

let calculatedWidth = ceil(CGRectGetWidth(calculatedFrame))
let calculatedHeight = ceil(CGRectGetHeight(calculatedFrame))
let finalHeight = (calculatedHeight + edgeInsets.top + edgeInsets.bottom)
rect.size = CGSize(width: calculatedWidth, height: finalHeight)

The ceil() should raise fractional mathematical value to a renderable screen value. The entire code is below.

class RoundedRectLabel: UILabel {
    let edgeInsets = UIEdgeInsets(top: 10, left: 20, bottom: 10, right: 20)

    override func textRectForBounds(bounds: CGRect, limitedToNumberOfLines numberOfLines: Int) -> CGRect {
        var rect = super.textRectForBounds(bounds, limitedToNumberOfLines: numberOfLines)
        if let text = text {
            let estimatedWidth = CGRectGetWidth(rect) - (2 * (edgeInsets.left + edgeInsets.right))
            let estimatedHeight = CGFloat.max
            let calculatedFrame = NSString(string: text).boundingRectWithSize(CGSize(width: estimatedWidth, height: estimatedHeight), options: .UsesLineFragmentOrigin, attributes: [NSFontAttributeName: font], context: nil)
            let calculatedWidth = ceil(CGRectGetWidth(calculatedFrame))
            let calculatedHeight = ceil(CGRectGetHeight(calculatedFrame))
            let finalHeight = (calculatedHeight + edgeInsets.top + edgeInsets.bottom)
            rect.size = CGSize(width: calculatedWidth, height: finalHeight)
        }

        return rect
    }

    override func drawTextInRect(rect: CGRect) {
        let textRect = UIEdgeInsetsInsetRect(rect, edgeInsets)
        super.drawTextInRect(textRect)
    }
}

To finish it off, we just need some custom drawing code. Feel free to draw whatever you like.

override func drawTextInRect(rect: CGRect) {
    UIColor.cyanColor().setFill()
    UIColor.blackColor().setStroke()

    let edgePath = UIBezierPath(roundedRect: rect, cornerRadius: 50)
    edgePath.lineWidth = 5.0
    edgePath.lineJoinStyle = kCGLineJoinRound
    edgePath.fill()
    edgePath.stroke()

    let textRect = UIEdgeInsetsInsetRect(rect, edgeInsets)
    super.drawTextInRect(textRect)
}

The entire code is also available at github.

View everything as optional

Macleod Sawyer | Web Portal Feed — Sat, 16 May 2015 03:00:00 GMT

Every week I throw up a quote or a saying that comes to mind on this whiteboard that sits on my desk.

This week it happens to be “View everything as optional”. I have no clue where I came up with it (if I did) or if I saw it online or in a book sometime in the past.

Now, what do I mean when I say “view everything as optional”? I am talking about your decisions, your life, everything around you. Did you have to get that coffee? Buy that shirt? Go out with friends? Tell your friends you weren’t going out because of other pressing matters (like watching Daredevil all night as I did)?

Everyone says that saying “No” is the best option. Truth be told that advice is horrible. You shouldn’t always say no to every question thrown at you, you won’t go anywhere with that, you just can’t be successful that way, you’ll have to say yes eventually.

Instead, think of everything as optional. Think of everything in your life, think of what could’ve been done better if you had just thought about it a little bit more. Don’t just immediately say no because that’s what everyone is pressuring you into saying and especially don’t just immediately say yes because that’s what everyone is pressuring you into saying. Instead, think beyond that, maybe it doesn’t deserve an answer, maybe you honestly just don’t care enough, maybe you’ll realize it isn’t vital to your life. Think of everything as optional.

Think of every time you have ever had to answer something, every time someone has told you something is final, that it can’t be fixed, that it just “isn’t possible”, or that you’re “getting ahead of yourself”. Take what they say under consideration, but realize nothing is final when making decisions, realize that everything around you is completely optional.

Explore your options. Don’t let the world tell you that you only have one option, don’t let them tell you what to think, don’t let them think for you. Be yourself, and make your own decisions. I realize sometimes it doesn’t work out that way, that some situations just “can’t” be optional, that what has happened, has happened. But, 99.9% of the time it’s “final” and you “can’t do anything about it” is just wrong in the scale of things.

If you set your mind to it, there is always an option. There is always something you can do. There is always another option that you’re just not thinking about. Even if everything seems empty as if you have no other choices as if you have been cornered as if you can’t go any farther. There is always an option.

Just look further beyond the questions, the answer is there. I promise. If you just think about it a little bit longer than that split second before giving the “final” answer, you will come to an option that is better and fits more with want you to want than what the other side that is pressuring you wants more. When I say look beyond, I mean it. Take the question apart word by word, take the situation apart and throw away everything you know and look at the events that caused it.

Life is not an exam.

There are no multiple choice forms in life, there are no exams. There are options beyond the normal “A” and “B”, and well beyond “C”, “D”, “E”, and “F”. Those options on tests from high school were thrown at you because those could be an answer. Within one of those questions could be the answer that you need to get that perfect score.

But what happens if the teacher made an error? What happens if that specific question didn’t have the correct answer to it, and you just knew, you just had this gut feeling that you were right and that the test was wrong? Would you speak up? That’s up to you because everything is optional.

You could just leave it be, you could just fill in the standard back up answer of “C” or you could think harder and ask the people around you, consult the teacher, consult your parents, the internet, textbooks, videos, professors, doesn’t matter as long as you just look beyond.

If you went for the “C” option you wouldn’t enjoy finding out that later that your gut feeling was right, and that you could’ve said something about it.

You fell into what society has told you – that there are only predetermined answers – to stop looking farther down the rabbit hole and find the answer that you need to fulfill yourself. Think of all those chances you could’ve had if you had just looked into the question more. It will fuel your doubt, it certainly fuels mine.

But, the chance at being more right instead of wrong because you looked into it because you went beyond the question and found a new option that wasn’t on the board, will certainly make yourself feel better and that’s because it is better.

Thinking for yourself, making your own choices, risking everything on a choice you feel is right, from all the research you have done is far more exciting and wonderful than letting the world tell you what your options are.

Sometimes the world is better at answering questions, sometimes they can make the better calls. But, sometimes they just can’t. I’m not telling you to always overlook what the world is telling you. Take them into consideration, look into them, but also look into the options that can’t be seen, the options that sit beyond the horizon of the black hole that we just can’t, that we just shouldn’t look into.

Find your own answer, don’t let yourself let the world find your answer. Be your own person. Have your own personality. Have your own choices.

View everything as optional.

Video - Storage Cluster Failover Demo

smcleod.net — Thu, 14 May 2015 03:00:00 GMT

A brief demonstration of the failover and recovery process on the storage clusters I’ve been building.

How to install Percona MySQL 5.6 on Ubuntu 14.04 LTS from a script (i.e. without an interactive prompt for the mysql root password)

Databases & Life — Thu, 14 May 2015 03:00:00 GMT

Ubuntu 14.04 (and various versions of Debian) come with MySQL 5.5. If you need MySQL 5.6 features (such as better join algorithms) you can install the Percona software, as described here.

However, the installation process prompts you for the database’s root password. This isn’t convenient if you’re running the installation process from a script e.g. a Vagrant or Docker initialization script. Here’s the whole script to install Percona 5.6 and to set the root password to “root” without a prompt:

Futility by Wilfred Owen

ho.dges.online — Wed, 13 May 2015 03:00:00 GMT

I remember reading this at school. It was the first WWI poem I think I read and I remember being very moved by it.
Move him into the sun—
Gently its touch awoke him once,
At home, whispering of fields half-sown.
Always it woke him, even in France,
Until this morning and this snow.
If anything might rouse him now
The kind old sun will know.

Think how it wakes the seeds—
Woke once the clays of a cold star.
Are limbs, so dear-achieved, are sides
Full-nerved, still warm, too hard to stir?
Was it for this the clay grew tall?
—O what made fatuous sunbeams toil
To break earth’s sleep at all?

LG Vertretungsplan App for Android

Niklas Korz — Tue, 12 May 2015 01:00:00 GMT

The last few weeks, I’ve been developing an app for the substitution plan of my school (hence the name: “Vertretungsplan” is German for substitution plan).

Redirecting Requests with NGinx: Using NGinx configuration to quickly redirect traffic

Joshua Rogers — Sun, 10 May 2015 15:00:00 GMT

A while back, I announced that I had moved my blog over to HTTPS. What about HTTP though? Do I serve content over HTTP still, or did I doom existing links to be forever broken? As you could guess by the title of this post, neither of those was satisfactory. Instead, I added four lines to our NGinx configuration telling it to listen on port 80 and respond to every request with a permanent redirect pointing to the same path under https://joshuarogers.

Marvel's Daredevil: Season 1

🦉 olshansky 🦁 — Sat, 09 May 2015 08:29:46 GMT

Olshansky's review of Marvel's Daredevil: Season 1

About TextUtils.

Prasham H Trivedi — Sat, 09 May 2015 03:00:00 GMT

Android SDK has an awesome class called TextUtils. It has many great methods that I use daily, and some magical methods as well.

Upsert lands in PostgreSQL 9.5 – A first look

CRAIG KERSTIENS — Fri, 08 May 2015 23:55:56 GMT

If you’ve followed anything I’ve written about Postgres, you know that I’m a fan. At the same time you know that there’s been one feature that so many other databases have, which Postgres lacks and it causes a huge amount of angst for not being in Postgres… Upsert. Well the day has come, it’s finally committed and will be available in Postgres 9.5.

Sure we’re still several months away from Postgres 9.5 being released, anywhere from 3-6 months as a best guess. That doesn’t mean we can’t take a first look at this feature. Though before we get into it a few special call outs of thanks to Peter Geoghegan of the Heroku Postgres team for being the primary author on it, Andres Freund who recently just joined Citus Data for his heavy contributions, and Heikki Linnakangas as well for his contributions.

And now onto the exploration. Upsert is the common name, but if you’re unfamiliar upsert is essentially create or update – Create this new record, but if a conflict exists update it. Let’s take a practical example.

Assume you have a web scraper that imports product information into a table. Each product has a UPC code, title, description, and link. There’s a unique constraint on the UPC code. Now, if your web scraper tries to insert a new product, and a product with the same UPC already exists, you’d usually get an error. But you don’t want the query to fail, you’d want to update the existing product instead. Maybe with a new image, maybe a new description, whatever have you, but I don’t want it to blow up… I simply want to capture the new data and save it.

So before: Insert a record… Exception this violates a unique constraint… Let your app figure out what to do. protip: often applications would try to work around this, but you can run a chance of a race condition and duplicate records if there’s a conflict. TLDR; it’s not a perfect solution.

Now: Insert a record… There’s a unique constraint violation… Okay, let’s just update all the new record’s fields inside a single transaction

So enough explanation, here’s how it actually looks in the syntax:

INSERT INTO products (
upc,
title,
description,
link)
VALUES (
123456789,
‘Figment #1 of 5’,
‘THE NEXT DISNEY ADVENTURE IS HERE - STARRING ONE OF DISNEY'S MOST POPULAR CHARACTERS! ’,
‘http://www.amazon.com/dp/B00KGJVRNE?tag=mypred-20’
)
ON CONFLICT DO UPDATE SET description=excluded.description;

It’s been a long time coming for this, and it makes building applications that need this kind of behavior even easier. While it would have been great for this to be available years ago, kudos to Postgres and its community for taking the approach that is safe for your data. The result we have now both provides the desired behavior of create or update, and is performant without the risk of race conditions for your data.

On my trip to Microsoft Build 2015

Nicola Iarocci — Fri, 08 May 2015 03:00:00 GMT

Last week I was in San Francisco for Build 2015, the annual conference event held by Microsoft and aimed toward software and web developers. Overall it has been a great experience. I especially enjoyed the opportunity to speak with Microsoft executives, Program Managers and developers about the new stuff they are cooking up and, in general, about that New Microsoft we all have been experiencing as of late (I have a guest post on MSDN Italy on that topic).

Tomorrowland

🦉 olshansky 🦁 — Wed, 06 May 2015 17:56:00 GMT

Olshansky's review of Tomorrowland

The 7 Ways to Earn a Living

one mikro2nd — Wed, 06 May 2015 13:40:05 GMT

“Work is love made visible” wrote Khalil Gibran. “If you cannot work with love but only with distaste, it is better that you should leave your work and sit at the gate of the temple, taking alms of those who work with joy.” In general there are 7 (maybe 8) ways of earning money: Traders A trader makes their living buying stuff and selling it for a higher price. Generally some value has to be added to the stuff for this to work, and especially in this highly connected, information-rich age when anybody can just fire up their web browser to find the manufacturer of the stuff you’re selling and buy it directly.

Write to temp iOS snippet

xenodium.com @alvaro — Wed, 06 May 2015 03:00:00 GMT

NSString *tempDir = NSTemporaryDirectory();
NSLog(@"%@\n", tempDir);
NSString *dataFilePath = [tempDir stringByAppendingPathComponent:@"my.file"];
[data writeToFile:dataFilePath atomically:YES];

Finding duplicate JavaScript code

Posts by Petr Stribny — Wed, 06 May 2015 00:53:44 GMT

View On Social Changes In 2015

Venam's Blog — Patrick Louis (Lebanon) — Wed, 06 May 2015 00:00:00 GMT

Hello fellow readers, This post is about an analysis of social changes in 2015.

Migrating to Laravel 5

Liza Shulyayeva — Tue, 05 May 2015 23:32:12 GMT

A few days ago I decided that there’s no use putting it off any longer - it’s time to upgrade to Laravel 5. I’ve been sitting on 4.2 for months and 5 is a major revision to…well…everything.

Starving

trv.rs — Tue, 05 May 2015 03:00:00 GMT

Heavenly Father, thank you for your graciousness in creating our bodies and our souls to mirror each other in some ways... your patterns in creation are like your fingerprint, allowing us foolish men and women to discover your hand in the physical world, and to better understand our spiritual selves. As people we are born, we drink milk, are weaned, mature, eat meat, reproduce... and the same words and concepts you have used for our spiritual lives as well.

When a sailor falls into the icy cold Atlantic, miles from rescue, the water slowly lowers his body temperature to deadly levels. His experience will slide from simple cold, intense shivering, incredible pain, the ironic sensation of burning, numbness... But the moment that he is most in peril of death is a slowly spreading feeling of bliss, that all is well, and he has nothing to fear. The cold floats away, and soon, so will his mind. A similar shutdown occurs to those dying of thirst. The most horrifying moment is when the sufferer no longer feels thirsty. The end is very near.

In my brief life, I have never been at the precipice of death by cold or thirst, but I have found that this is has certainly been true of my foolish soul. The times that I have found that I feel like I don't need to drink of the Word are the moments I am most in peril of spiritual dehydration... and I confess that these moments are far too common.

Heavenly Father... please bring me to your Word. I want to partake daily, hourly, constantly. I know I'm promised that the more I drink, the more I'll feel the thirst for your Truth in a Holy addiction. Keep me from the pride of thinking that I'm doing well and can skip this morning, I know that this is a dangerous time, near to exhaustion. Please draw me near to Yourself, and grant me a sweet time of communion and encouragement. In your Son's precious name I pray, amen.

Greece travel bookmarks

xenodium.com @alvaro — Mon, 04 May 2015 03:00:00 GMT

Sri Lanka travel bookmarks

xenodium.com @alvaro — Mon, 04 May 2015 03:00:00 GMT

Switzerland travel bookmarks

xenodium.com @alvaro — Mon, 04 May 2015 03:00:00 GMT

Thailand travel bookmarks

xenodium.com @alvaro — Mon, 04 May 2015 03:00:00 GMT

Madagascar travel bookmarks

xenodium.com @alvaro — Mon, 04 May 2015 03:00:00 GMT

Avenue of the Baobabs.

Hong Kong travel bookmarks

xenodium.com @alvaro — Mon, 04 May 2015 03:00:00 GMT

Causeway Bay pedestrian crossing.
Hong Kong's most breathtaking views: where to glimpse the city from above.
Hotel Icon (allegedly amazing service/extras).
Lin Heung Tea House (bakery/tea/dim sum).
Mak's Noodle (Jordan).
Siu yuk (Roasted Pig).
Tsang Tsou Choi (King of Hong Kong).
Tsim Chai Kee (recommended over Mak's wonton/soup).

Barcelona travel bookmarks

xenodium.com @alvaro — Mon, 04 May 2015 03:00:00 GMT

Iceland travel bookmarks

xenodium.com @alvaro — Mon, 04 May 2015 03:00:00 GMT

An Iceland travel log.
Apartmenthouse.is for local flats.
Blue lagoon spa.
citywalk.is: Free walking tour.
Design March.
Drive it yourself: The Snæfellsnes peninsula.
Efstidalur: farm to table restaurant.
Eight must-see spots in Iceland's wild west.
Fridrikv restaurant.
Gangleri outfitters.
Golden circle's waterfalls and geysers.
grillmarkadurinn.is restaurant.
Horse farm hotel.
Hotel Budir.
How to have a budget break in iceland.
Iceland itinerary.
Lebowski Bar seriously?
Noodle Station.
Pink Iceland tours.
Reykjavik Apartment Hotel – Accommodation in City Centre (swanhouse.is).
Reykjavik City Breaks (Book now with British Airways).
Sandholt.
South coast's waterfalls and caves.

The Little MongoDB Book 2nd Italian Edition

Nicola Iarocci — Mon, 04 May 2015 03:00:00 GMT

The second edition of the Little MongoDB Book Italian Edition is finally up. It is aligned to MongoDB 2.6 and includes a much needed introduction to the Aggregation Frameworks. Enjoy.

Contributing to open source projects.

Prasham H Trivedi — Sun, 03 May 2015 03:00:00 GMT

With open source nature of android and it’s vast open source ecosystem, we often (want to) contribute to an open source library. Here I have tried to cover some points to be kept in mind for library developers and contributers.

How to Learn Vim: Vim Refcard

Mike Kasberg's Blog — Sat, 02 May 2015 22:00:00 GMT

Vim Refcard on Github

I’m a huge fan of reference cards like the one Emacs provides. In fact, I think that a good reference card is probably the best way to learn a new software program. Whether it’s Vim or anything else.

Do you want to learn Vim? Print out this PDF, put it next to your monitor, and start using Vim whenever you need a text editor. Seriously. Just do it. It will be slow at first. In fact, it might be a little painful. But after an hour or two, you’ll be cruising along like a seasoned vet. You’ll be amazed how well you can remember the shortcuts - your brain can remember them because you were using them.

The Emacs refcard is really well done, but I couldn’t find a similar one for Vim. This was disappointing because I know how useful a good reference card is. So I busted out my LaTeX editor and made one. It’s open source, and available on Github. Feel free to open a pull request if you think it’s missing anything important.

The Boy Next Door

🦉 olshansky 🦁 — Sat, 02 May 2015 13:02:54 GMT

Olshansky's review of The Boy Next Door

Kubernetes clusters with Oh-My-Vagrant

The Technical Blog of James on purpleidea.com — Sat, 02 May 2015 03:07:11 GMT

I’ve added the ability to deploy a Kubernetes cluster with Oh-My-Vagrant (omv). I’ve also built an automated developer experience so that you can test your Kubernetes powered app in minutes. If you want to redeploy a new version, or see how your app behaves during a rolling update, you can use omv to test this out in minutes! I’ve recorded a screencast (~15 min), if you’d like to see some of this in action.

9 Truths Computer Programmers Know That Most People Don't.

Macleod Sawyer | Web Portal Feed — Fri, 01 May 2015 17:51:21 GMT

I wrote this when I was sixteen. While I do believe that most of what is written is still accurate, be warned that it’s intended audiences were those most susceptable to clickbait. It worked. This article was archived from my past blogs, comments have disapeared sadly but you can find quite a few of them here on HackerNews where it remained on the front page for several days. Simpler times.

Computer programmers know far more about computers and code than the average person does, and honestly some of it’s scary.

# Fact #1

“Under the hood, most critical software you use every day (like Mac OS X, or Facebook) contains a terrifying number of hacks and shortcuts that happen to barely fit together into a working whole. It would be like taking apart a brand-new 747 and discovering that the fuel line is held in place by a coat-hanger and the landing gear is attached with duct tape.” — Ben Cherry

That’s the funny thing about code, the website or program may work beautifully, it may run smoothly, and it may be absolutely beautiful on the front-end side (what the user sees). But, behind everything that makes it work it will have so many errors, and work arounds that barely work and that shouldn’t work, but do for some strange reason.

# Fact #2

"That about 25% of the hours spent writing an application are spent figuring out ways the end user will do something wrong.” — Brian Humes

Everytime we build something, we have to sit back and think of how the end-user will end up fucking it up. What they will click on, what they will write, the phrasing of questions, the language used, and how what we write could be interpreted differently. If we wrote the code like how we would use the project, well then there will be so many issues because we know how the program works, and the end user doesn’t.

# Fact #3

“A programmer is not a PC repair man.” — Ritesh Kumar Gupta

A programmer is one who deals with algorithms and design principles, not the one who repairs a computer. We may know how the internal workings of a computer work, how code fits together (or rather hacked together as I explained in Fact #1). But, that does not mean we know how to fix hardware. That does not mean we know how to fix that issue you’re having with chrome that makes it crash everytime you open it, or why you’re computer is always overheating and the battery dying. Computers programmers, at the least know how to program computers, not fix them.

# Fact #4

“Programming is thinking, not typing.” — Casey Patton

Most of programming is spent sleeping, walking around, staring out the window, or doing anything else that helps you relax and think. Relaxing is a major key to programming, its not just sitting down and writing a thousand or more lines or code, and pushing out a program or app. We have to sit down, walk around, and just think. We need to think about how to come up with the concept, fix the issues with it, find a way to make it work, how it’s going to work. Relaxation is the only way we can fix the issues the best way we can.

# Fact #5

Counting starts from zero, not one.

This is important in every programmers life. Counting starts at 0, your “1" is my “0", your “10" is my “9". The reason why this is because computer programming is all about efficiency, and even small improvements in efficiency can make big differences at scale.

And yes, counting from zero is slightly more efficient than starting at 1. Computers are built on a 0 and 1 numbering system that makes up everything (hello binary!). Counting from 0, is just easier and creates efficiency. (you can read more here)

# Fact #6

“Programming is best done “in the zone” — a (pleasant) state of mind where your focus on the task is absolute and everything seems easy. This is probably much like “the zone” for musicians and athletes.” — Morgan Johansson

Ever wondered why programmers are known as night birds? Why we stay up all night? Because it allows us to get into the zone, it allows us to focus on one thing and not have to worry about being interrupted by someone — because they are all asleep. It’s a long stretch of the day where no one is up and no one is calling or trying to talk to us. It’s a great time to program, and think.

# Fact #7

Sleeping with a problem, can actually solve it.

If you have a problem you are told to sleep on it, forget it, put your mind at rest. But, with programmers its the go to way to solve the problem not because it gets us away from it, but because it for whatever reason helps us solve the problem with our code. Many times I have come across an issue, spent hours and hours of work on it, just trying to fix what should should be a simple problem with a simple fix. But, by going to sleep for 20 minutes, an hour, six hours, twelve hours, we can wake up and immediately know the answer to the problem.

# Fact #8

A parent may kill its children if the task assigned to them is no longer needed

Not something you would like to hear someone say while out, now is it? (and if it is, you should seriously check into a mental health facility, or turn yourself into your local police department).

It’s not as gory as one would think, programs are written like a hierarchy. With the parent managing the processes below them.

An example of the parent-child relationship.

When a parent no longer needs a child they kill it, meaning when a program no longer needs to do something (say send an email), they kill the connections to the server as its not needed, basically killing the child.

# And finally, fact #9

Just as you’re usually not impressed when we brag about how much we know about computers, we’re not impressed when you brag about how little you know about them.

Seriously. Just stop, please it’s annoying. We don’t care. We really really don’t care how proud you are of not wanting to learn new things. Now, its understandable if you are just saying “I don’t know much about computers” or “I’m not really interested in computer programming” but bragging about how much you dont know about computers is just annoying. Stop.

This article was archived from my past blogs, comments have disapeared sadly but you can find quite a few of them here on HackerNews: https://news.ycombinator.com/item?id=9164665 3

Git with GitHub the Simple Way

Jim Westergren — Thu, 30 Apr 2015 19:03:00 GMT

This is what is working for me and may not be the best for you. This is very beginner level. I am using the Git Bash from here. Starting a new repo Create the repo in GitHub, include readme. Create a team in Github and make sure you have access to the repo. Create a [...]

Building clang-format

xenodium.com @alvaro — Thu, 30 Apr 2015 03:00:00 GMT

Based on instructions from Building clang-format and friends on OSX Mountain Lion.

#!/bin/bash
  set -o nounset
  set -o errexit

  # Based on instructions from:
  # http://blog.hardcodes.de/articles/63/building-clang-format-and-friends-on-osx-mountain-lion

  readonly LLVM_DIR_PATH='/tmp/llvm'

  update_repo() {
    if [[ ! -d $1 ]]; then
      git clone $2
    else
      cd $1
      git pull
      cd ..
    fi
    cd ..
  }

  update_all_repos() {
    update_repo "llvm" "http://llvm.org/git/llvm.git"
    pushd "${LLVM_DIR_PATH}/llvm/tools"
    update_repo "clang" "http://llvm.org/git/clang.git"
    popd
    cd "../../${LLVM_DIR_PATH}/llvm/tools/clang/tools"
    update_repo "clang-tools-extra" "http://llvm.org/git/clang-tools-extra.git"
    cd "../../.."
  }

  build_clang() {
    mkdir -p clang
    mkdir -p build
    cd clang
    ../llvm/configure --enable-libcpp --enable-cxx11 --enable-debug-symbols=no --enable-optimized --prefix="${LLVM_DIR_PATH}/build"
    make install
  }

  mkdir -p $LLVM_DIR_PATH
  cd ${LLVM_DIR_PATH}
  update_all_repos
  build_clang

Bonus: use clang-format-configurator.

Programmatic iOS Auto Layout

xenodium.com @alvaro — Thu, 30 Apr 2015 03:00:00 GMT

Basic iOS auto layout usage. See Adopting Auto Layout and Visual Format language for reference.

- (instancetype)initWithFrame:(CGRect)frame {
  self = [super initWithFrame:frame];
  if (self) {
    // Disable autoresizing mask translation for parent.
    self.translatesAutoresizingMaskIntoConstraints = NO;

    _subview1 = [[UIView alloc] init];
    // Disable autoresizing mask translation for subview.
    _subview1.translatesAutoresizingMaskIntoConstraints = NO;

    _subview1.backgroundColor = [UIColor redColor];
    [self addSubview:_subview1];

    // Creates a dictionary of bindings to be used in visual format.
    NSDictionary *viewBindings = NSDictionaryOfVariableBindings(_subview1);

    // H: horizontal layout
    // |-50- spacing in relation to superview
    // [_subview1(==50)] subview1's width
    [self addConstraints:[NSLayoutConstraint constraintsWithVisualFormat:@"H:|-50-[_subview1(==50)]"
                                                                 options:0
                                                                 metrics:nil
                                                                   views:viewBindings]];
    [self addConstraints:[NSLayoutConstraint constraintsWithVisualFormat:@"V:[_subview1(==50)]"
                                                                 options:0
                                                                 metrics:nil
                                                                   views:viewBindings]];
  }
  return self;
}

Also consider:

A UIView Subclass should implement intrinsicContentSize.
A UIView Subclass should never add constraints on neither itself (ie. self) nor superview.

References

Auto layout best practices for minimum pain.

Avoid adding "drive-by formatting changes" to commits

Databases & Life — Thu, 30 Apr 2015 03:00:00 GMT

Recently, when reviewing code, I saw a commit resembling the following:

- if (x) foo();
- if (y) bar();
+ if (y) {
+    bar();
+ }

It’s not easy to see that of the two “if” statements, only one was actually deleted. The other had its formatting changed, but was otherwise not altered.

To prevent this, go through the “diff” before doing the commit. Revert any changes which have happened which haven’t changed the functionality of the code.

Google Summer of Code 2015!

spinda's dreamwidth — Wed, 29 Apr 2015 06:57:27 GMT

My proposal for the Haskell's Google Summer of Code 2015 has been accepted! I'm super excited for this. There's a list of the other accepted proposals on reddit, and I've posted my full proposal as a Gist. I'll be working on embedding LiquidHaskell signatures in Haskell's native type system. Woo!

comments

More AUR Packages!

spinda's dreamwidth — Wed, 29 Apr 2015 04:02:02 GMT

New Packages

nemo-owncloud-git, for my port of the Nautilus ownCloud plugin to Nemo

Adopted Packages

gargoyle, an interactive fiction player
nemo-dropbox-git, Dropbox plugin for Nemo
nemo-fileroller-git, File Roller plugin for Nemo
jips, an IPS patcher

comments

MacBook 2015 Revisited

Mike Schaeffer — Wed, 29 Apr 2015 03:00:00 GMT

Since my last post, I dropped by an Apple Store to take a look at the 2015 MacBook. It is difficult to overstate how startlingly small the new machine is in person. I may be biased by the internal specifications, but the impression is much more 'big tablet' than 'small laptop'. The other standout feature was the touchpad. It continues Apple's tradition of high-quality touchpad implementations, removes the mechanicical switch and hinge, and adds force sensititivy and haptic feedback. The mechanical simplifications alone are a worthwhile improvement.

I also spent some time typing on the keyboard. It's as shallow as you'd think, but the keys are very direct have a positive feel. There's none of the subtle rattling found on most small keyboards and it registered every keypress. I'm not completely convinced yet, but it at least seems possible that this type of keyboard could become the preferred keyboard for some typists.

The performance of the machine is also a point of interest. Even the lightly loaded demo machine on the showroom floor had a few hiccups paging the display around from one virtual desktop to the next. Maybe it's nothing, but it does make me wonder if the machine can keep up with daily use, particuarly after a few OSX updates have been released. (For me, I think it'd be fine, but I spend most my time in Terminal, Emacs, and Safari, none of which are exactly heavy-hitters.)

Implementing a simple event system in C++11

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Sun, 26 Apr 2015 01:03:53 GMT

I recently started thinking about event handling systems. I have been using Boost.Signals for some time now and was pretty happy with it. Then the number of signals spiralled out of control and forced me to re-think the design. The venerable design patterns book offers the Observer pattern or its modified form, the Publish–subscribe pattern. Both design patterns involve passing information between possibly unrelated objects.

My goal was to come up with a system that couples objects very loosely. Ideally, the central instance, which I am going to call dispatcher because it makes Tamiko happy, should know barely anything about an event it delivers. Using C++11, I came up with a rather simple and straightforward design.

The following discussion will leave out unsubscriptions by observers because it makes the code samples harder to follow. The code on GitHub, though, has this functionality.

A base event class

Regardless of how anything else is implemented, we need a base class for describing an event. Mine looks like this:

class Event
{
public:
  virtual~ Event();

  using DescriptorType const char*;

  virtual DescriptorType type() const = 0;
};

Not much to see here. The destructor is kept virtual because Event is a polymorphic base class. Every derived event needs to implement the type() function to describe itself. The DescriptorType refers to const char*, meaning that each event uses a const char* to create a unique identifier. In practice, this is what it might look like:

class DemoEvent : public Event
{
public:
  DemoEvent();
  virtual ~DemoEvent();

  static constexpr DescriptorType descriptor = "DemoEvent";

  virtual DescriptorType type() const
  {
    return descriptor;
  }
};

Note that the DemoEvent class also offers a static constexpr descriptor. This permits us to write DemoEvent::descriptor when referring to the descriptor of this class. In other words, we do not need to instantiate anything here.

The dispatcher

The role of the dispatcher is to manage multiple observers that are interested in a certain event. To this end, we need a predefined interface every observer needs to implement. C++11 offers std::function for this purpose. We also need to keep track of which observers are interested in which event. This is where the DescriptorType of events comes in. Since DescriptorType is a const char*, we can use it as key in a map! Whenever an event occurs, we then look up its corresponding type, use this to access the map, and call all observers.

This is the basic interface of the dispatcher:

class Dispatcher
{
public:

  using SlotType = std::function< void( const Event& ) >;

  void subscribe( const Event::DescriptorType& descriptor, SlotType&& slot );

  void post( const Event& event ) const;

private:

 std::map< Event::DescriptorType, std::vector<SlotType> > _observers;
};

Using post() any client can send events to all observers that are interested in them. The dispatcher keeps track of slots to call using the map. The implementation turns out to be surprisingly simple:

void Dispatcher::subscribe( const Event::DescriptorType& descriptor, SlotType&& slot )
{
  _observers[descriptor].push_back( slot );
}

void Dispatcher::post( const Event& event ) const
{
  auto type = event.type();

  // Ignore events for which we do not have an observer (yet).
  if( _observers.find( type ) == _observers.end() )
    return;

  auto&& observers = _observers.at( type );

  for( auto&& observer : observers )
    observer( event );
}

Defining observers

So, how do we use the dispatcher then? By invoking the magic of std::bind. Let us first define a simple observer class. In this class, we can also check the underlying event type, for example if we want to react differently to certain events:

class ClassObserver
{
public:
  void handle( const Event& e )
  {
    if( e.type() == DemoEvent::descriptor )
    {
      // This demonstrates how to obtain the underlying event type in case a
      // slot is set up to handle multiple events of different types.
      const DemoEvent& demoEvent = static_cast<const DemoEvent&>( e );
      std::cout << __PRETTY_FUNCTION__ << ": " << demoEvent.type() << std::endl;
    }
  }
};

Again, we can see that the static constexpr comes in handy. We do not need an instance of DemoEvent here. Now, assuming we have a dispatcher variable called dispatcher, which is very creative, we can finally use std::bind():

int main( int, char** )
{
  using namespace std::placeholders;

  ClassObserver classObserver;
  Dispatcher dispatcher;

  dispatcher.subscribe( DemoEvent::descriptor,
                        std::bind( &ClassObserver::handle, classObserver, _1 ) );

  dispatcher.post( DemoEvent() );
}

This also demonstrates the basics of posting an event. It is probably not apparent in this example, but the dispatcher does not need to know anything about events excepts that they are inheriting from the same base class. We can pass arbitrarily complex objects to and fro without changing the coupling between the dispatcher and the observers.

Limitations

A severe limitation of the design in this form is the inability to handle events in different threads. The post() interface assumes that it gets passed a const reference to an event. This makes sense insofar as we do not know how "heavy" an event is—or whether it makes sense to be copied. Thus, in case one observer chooses to handle an event in a separate thread, the event could potentially be destroyed before the observer finished handling it. One solution for this problem would be to use std::shared_ptr to pass events around. This at least would solve the destruction issue—but I am not very firm concerning the thread safety of std::shared_ptr.

Another issue is a systematic problem: I cannot guarantee the uniqueness of the self-described events. If a client chooses to implement events A and B and both return, say "Foo" as their event type, the dispatcher will not be able to separate between these events. Currently, I do not have a solution for this—hence, comments would be highly appreciated!

Update (2021-01-31): An issue in the GitHub repository belonging to this post discusses a potential extension to handle different event types by means of class templates in the dispatcher class. This is a very neat suggestion—check it out.

The code

A slightly more involved implementation is available as the "Events" repository in my GitHub account. The code is licensed under an MIT license.

I hope reading this was both insight- and eventful for you.

House of Cards: Season 3

🦉 olshansky 🦁 — Sat, 25 Apr 2015 09:17:36 GMT

Olshansky's review of House of Cards: Season 3

Time Limits

John Jacobsen — Sat, 25 Apr 2015 03:00:00 GMT

Picking up the art blogging again after a long hiatus.

Recently I’ve been thinking about constraints a lot, and finding out how helpful they are. I think they are especially helpful for artists.

For example, since I started my new job last year, I am in the habit of taking photos on the way to or from work. I use the following constraints:

Use only the iPhone with no extra lenses (digital filters/manipulation OK, but only on the iPhone itself).
Picture must be taken on the way to or from work, or near the office.
Post the ones I like on Facebook, to this album ("Views from the Commute").

The pictures are not always awesome, but the practice itself is most satisfying. It’s a way to stay creative and visually-focused despite the demands of a “real” job. And I find the instant feedback I get from friends and family on Facebook more satisfying than many a gallery show has been.

Image from "Views from the Commute"

Another way I’ve been exploring constraints in the last few years is by painting outside. This practice has a number of constraints more or less built in: you can’t carry too much stuff around; it’s only light out for so long; and you typically paint the stuff you find in front of you (rather than stuff you make up).

Lately I’ve been redoing my Web sites and thinking about some old art projects as I sift through blog posts. I really enjoyed working with SketchUp awhile back and have been itching to return to that. So I’ve been wondering what sort of constraints might be helpful, similar to the ones I get “for free” by painting outside.

I decided on the following formula:

Four hour limit per painting:
- Hour one: make an image using SketchUp
- Hour two: transfer to panel and work on the drawing
- Third hour: underpainting
- Fourth hour: top layers / final painting.
Fifteen minute break in between hours
Use a Clojure program I wrote to help with the gridding of the drawing.

First stage, after one hour: exported from SketchUp and gridded

After second hour: drawing

After third hour: underpainting

Final image

Detail 1

Detail 2

Lessons:

Constraints help. I ordinarily have trouble finishing paintings. Though there were a few points when I thought things were totally off the rails and the work was going to be a disaster, overall I had a lot more focus (and fun!) than I often do while painting.
I originally planned hour three (underpainting) to be in acrylics; but I found I wanted to just jump in with oils, and it worked fairly well. Being flexible about the plan in this regard was good.
Having painting supports ready in advance was helpful; NOT having a concept or content queued up (for the SketchUp part) was also good, I think, though I might experiment with brainstorming content as a (timed) stage in the process.
SketchUp is a total rabbit hole. I could have burned the entire time just fooling around in SketchUp alone. Warning: dragons dwell there!

Advanced Oracle Troubleshooting Guide – Part 12: control file reads causing enq: SQ – contention waits?

Tanel Poder Blog — Sat, 25 Apr 2015 02:23:32 GMT

Vishal Desai systematically troubleshooted an interesting case where the initial symptoms of the problem showed a spike of enq: SQ – contention waits, but he dug deeper – and found the root cause to be quite different. He followed the blockers of waiting sessions manually to reach the root cause – and also used my @ash/ash_wait_chains.sql and @ash/event_hist.sql scripts to extract the same information more conveniently (note that he had modified the scripts to take AWR snap_ids as time range parameters instead of the usual date/timestamp):

Keeping track of your things

Venam's Blog — Patrick Louis (Lebanon) — Sat, 25 Apr 2015 00:00:00 GMT

Hello fellow nixers, This thread is about having your machine help you remember things throughout your days, to be your companion.

Working with YANG Models - A brief intro to 'pyangbind'

rob.sh — Sat, 25 Apr 2015 00:00:00 GMT

Both the Hawaii IETF meeting (IETF91) and the subsequent meeting we had a few weeks ago in Dallas were somewhat YANG-heavy. Following work to move towards YANG as a standard modelling language for network configuration, and the subsequent IESG statement effectively deprecating SNMP as the way that we present network configuration - the IETF, and especially the routing area, has dived head-first into YANG.

Indeed, I’ve been occupied somewhat with some really great collaborative work with a number of awesome engineers from Google, Microsoft, AT&T, Level3, Yahoo!, Facebook, Cox, Verizon and others on the OpenConfig initiative. We’re trying to take an operator and use-case driven approach to developing YANG modules for both configuration and defining the schema for telemetry. This work has turned up a few times in the press, and I should probably write something separate about it in the near future.

However, one observation that a number of people have made, is that there’s really limited tooling available to work with YANG modules. We have (the rather excellent) pyang, which provides a validation tool for YANG modules and the corresponding JNC plugin that creates Java classes – but after that, options start to run pretty dry for what one might use, other than commercial products such as tail-f NCS. In some cases, the way that these modules work is also a bit esoteric, requiring quite a lot of care around what the YANG types are in the consuming code.

To drive adoption of YANG and NETCONF for making the network more programmable – we need to make it easy to program the network. To this end, I started some work, with the aim of:

hexo init blog

Reilly Tucker Siemens — Fri, 24 Apr 2015 12:57:13 GMT

I've been wanting to start a blog for a long time now. Today I'm pulling the trigger on that with a simple hexo init blog. Well, it wasn't that simple, so I feel like it's worth talking about a few of the complications I had.

Hyperion

Code & Visuals — Fri, 24 Apr 2015 03:00:00 GMT

Just a quick update on future plans. Starting in July, I’m going to be working full time for Walt Disney Animation Studios as a software engineer on their custom, in-house Hyperion Renderer. I couldn’t be more excited about working with everyone on the Hyperion team; ever since the Sorted Deferred Shading paper was published two years ago, I’ve thought that the Hyperion team is doing some of the most interesting work there is in the rendering field right now.

I owe an enormous thanks to everyone that’s advised and supported and encouraged me to continue exploring the rendering and graphics world. Thanks, Joe, Don, Peter, Tony, Mark, Christophe, Amy, Fran, Gabriel, Harmony, and everyone else!

Normally as a rule I only post images to this blog that I made or have a contribution to, but this time I’ll make an exception. Here’s one of my favorite stills from Big Hero 6, rendered entirely using Hyperion and lit by Angela McBride, a friend from PUPs 2011! Images like this one are an enormous source of inspiration to me, so I absolutely can’t wait to get started at Disney and help generate more gorgeous imagery like this!

Organizing Background Worker Queues

Brightball Articles — Thu, 23 Apr 2015 23:58:00 GMT

At work earlier today I ran across an issue where one of our application queues got backed up and it got me to thinking about how queues are organized in general. The TLDR answer: use urgency and intensity.

Learning Japanese bookmarks

xenodium.com @alvaro — Thu, 23 Apr 2015 03:00:00 GMT

Japan travel bookmarks

xenodium.com @alvaro — Thu, 23 Apr 2015 03:00:00 GMT

21 Awesome Things to Do in Yokohama, Japan (2020 Guide).
21 free things to do in Tokyo.
6 Tokyo Travel Tips to Help You Get Around the City | Tokyo Weekender.
7 day Japan Rail pass (first class?), possibly only sold outside Japan.
80/20 Japanese by Richard Webb - Publishizer.
A Guide to the regional ramen of Japan.
A guide to understanding "Small Seasons".
A Moss Girl’s Guide to Japanese Moss Viewing.
A no-sushi guide to food in Japan.
A trip to Japan (Quora answer).
An unsponsored review of the Tawaraya Ryokan in Kyoto, Japan.
Arashiyama Bamboo Grove.
Autumn Leaves in Hokkaido.
BBC - Travel - Japan’s special take on a packed lunch.
Bushido: Way of Total Bullshit.
COOL JAPAN(?)" - What are things about Japan that you think are actually cool?.
Drinking Japan.
Eki-stamps / 駅スタンプ | eli's japan blog.
Five Best: Japanese Ryokan.
Fuji Q Highland (rollercoaster theme park). Check out haunted hospital.
Furano, Hokaido.
Going to Kyushu, Japan? Why Visiting Yakushima is Worth it - The Ruby Ronin.
Guest houses: Kazariya, Rakuza, Musubian.
Hiiragiya ryokan.
Himeji Castle.
Hiroshima oysters (try the one with cheese).
Historic Villages of Shirakawa-gō and Gokayama - Wikipedia.
Hokkaido summer flowers.
Hypermedia (internal travel website).
jalan.net (travel booking site).
Japan guide.
Japan Info.
Japan is launching a program that will give away abandoned homes - INSIDER.
Japan Rail Pass.
Japan Study Program.
JAPAN TRAVEL DATA 15DAYS, 3.5GB on 4G/LTE- NANO SIM.
Japan's glut of abandoned homes: Hard to sell but bargains when opportunity knocks (The Japan Times).
Japanese beaches.
Japanese Foods Encounter Like No Others (food search).
Japanese rule of 7.
Japan’s 72 Microseasons.
Kawachi Fujien 河内藤園 (Kawachi Wisteria Garden).
Kodo – Preparing the Censor.
Kongō Sanmai-in at Kōya-san.
Koya Bound – Eight Days on the Kumano Kodo | Hacker News.
Koyasan (needs booking).
Landed Japan Book (local knowledge to buy Japanese real estate).
Lots of goodies. Bamboo forest, oh my.
Michael’s Cafe American – Black Ship Coffee.
Miyajima: One of the top three scenic spots in Japan.
Monument in front of the Shin-Yatsushiro Station.
Nagakushiyama park.
Nagoya.
Natadera Temple in winter.
Niseko Ski Resort.
Okinawa: secrets for a long and happy life.
Okonomimura (all okonomiyaki restaurants).
Onsen.
Quora: What are some of Japans best kept secrets?
Saitamaya: The Master of Grilled Meat - YouTube.
Satsuma Rebellion: Satsuma Clan Samurai Against the Imperial Japanese Army.
Shitamachi #Museum in Ueno and catch a glimpse into old-fashioned living.
Siege of Kumamoto Castle (Wikipedia).
Snapshots of Tokyo’s vivid street life (Hacker News).
Table of Content for japan2013.
Table of Content for japan2014.
Table of Content for japan2015.
TAKAZAWA Candle (or look for ikaragata shape candle).
The 2020 Hachinohe Enburi Festival Schedule (Sakura festival).
The village of living water.
Things to look out for in Japan (Quora).
Tohoku Travel Guide.
Try out sukiyaki.
Uncharted Tokyo.
Vitra | The Hill of the Buddha.
Zao Fox Village.
国指定重要文化財「野村別邸　碧雲荘」.
Neighbourhoods
- Mishuku
- Sancha neighborhood. Lots of places to eat.
- Sangenjaya
- Soba Noodle Kochi
- Seiki (Japanese dishes)
- Sent from my iPhone
- Sakae street
- Tokoshima (wine and yakitori)

Emacs key bindings and maps

xenodium.com @alvaro — Thu, 23 Apr 2015 03:00:00 GMT

based on masteringemacs.org.

bonus tip

prefix key, followed by c-h, lists keys in prefix.

keymap

maps key to action.

keymap found in buffer and most major modes.

keys

undefined: self explanatory.
prefix key: ie. c-x (part of complete key).
complete key: complete input executes associated command.

mapping

(define-key keymap key def): add to current buffer map.
(local-set-key key command): add to active buffer (no map option).
(local-unset-key key)
(global-set-key key command): add to global keymap (all buffers).
(global-unset-key key)

key codes

kbd: macro transaltes human-readable key to emacs readable.
function and navigation keys must be surrounded by <>.
example: (kbd "c-c p") or (kbd "<f8>") of (kbd "<down>").

remapping

use remap to replace mapping (ie. kill-line with my/kill-line).
(define-key keymap [remap original-function] 'my-own-function).

reserved keys

"c-c ?" generally reserved for you, but third party packages use it.
function keys (ie. f1-f12).
hyper and super (ancient).

lookup order

in a nutshell: minor mode keys, local keys, global keys.
full order:
1. overriding-terminal-local-map: terminal-specific key binds.
2. overriding-local-map: override all other local keymaps (avoid if possible).
3. char property at point: useful for yasnippet.
4. emulation-mode-map-alists: advanced multi-mode keymap.
5. minor-mode-overriding-map-alist: minor modes in major modes.
6. minor-mode-map-alist: as previous (preferred for minor modes) <—–
7. current-local-map: buffers current local map.
8. current-global-map: last place to look (ie. global).

mode hooks

(local-set-key (kbd "c-c q") 'my-awesome-method)) in hook-method.
for key-chord-define, use current-local-map.

Matlab and Audio Files

bastibe.de — Wed, 22 Apr 2015 03:00:00 GMT

So I wanted to work with audio files in Matlab. In the past, Matlab could only do this with auread and wavread, which can read *.au and *.wav files. With 2012b, Matlab introduced audioread, which claims to support *.wav, *.ogg, *.flac, *.au, *.mp3, and *.mp4, and simultaneously deprecated auread and wavread.

Of these file formats, only *.au is capable of storing more than 4 Gb of audio data. But the documentation is actually wrong: audioread can actually read more data formats than documented: it reads *.w64, *.rf64, and *.caf no problem. And these can store more than 4 Gb as well.

It's just that, while audioread supports all of these nice file formats, audiowrite is more limited, and only supports *.wav, *.ogg, *.flac, and *.mp4. And it does not support any undocumented formats, either. So it seems that there is no way of writing files larger than 4 Gb. But for the time being, auwrite is still available, even though deprecated. I tried it, though, and it didn't finish writing 4.8 Gb in half an hour.

In other words, Matlab is incapable of writing audio files larger than 4 Gb. It just can't do it.

My weekend with Elm

λm.me — Tue, 21 Apr 2015 03:00:00 GMT

Ludum Dare is an online game jam competition/event that takes place every 4 months. Each competition starts when it's theme is announced and each participant has 48/72 hours for compo/jam entry respectively to complete and publish a game based on the theme.

This weekend LD32 took place, and I decided to participate while using Elm to build my game - Banana Bandit (source on github).

Edit: post-jam updates

Since I didn't have a lot of Elm experience (I wrote at most 100 LOC in it) prior to jam, I decided that my goal is to learn Elm, FRP and a new way to model games on the fly, during the jam, and gain real experience working with it, while somewhat sacrificing the quality of the end result (the game).

In this post I will try and describe my reflection of the weekend, what went wrong, what went well, and what could have been done better.

Starting the work was pretty easy, I was able to install Elm-platform pretty easily,

There are many examples on the main Elm website that can be run and changes online
There is an Elm architecture tutorial that explains how to start modeling applications in Elm
There is a Game Skeleton which I used as a template
Elm compiles to JavaScript, means it will work on the web - a huge win for LD in my opinion
There is a vim syntax-highlighting plugin
elm-reactor saves the trouble of compiling and running every time
I was able to start modeling and look at the changes to the model over time without drawing anything
Graphics were really easy to add, change and replace. From simple rectangle place holders to images and animations.

Problem is, since I started from scratch, I had to implement all the elements and logic of a game, game objects with position, velocity, state, actions, movement logic with acceleration and deceleration, collision detection, timers, interactions and so on. All those features that comes today with every game engine were very time consuming and were proving tricky to implement. Which is why such things need to be done prior to a jam. But I learned a lot implementing all of this things.

Timers changed too quickly because I was accidentally working with milliseconds instead of seconds
Movement and acceleration did bizarre things due to incorrect calculations
AI bugs due to calculation mistakes
Collision problems
Infinite recursion by accident that stuck the game
Maze generator logic bugs

And more.

Elm-reactor proved valuable in some of these bugs. Being able to watch values changing over time, stop and rewind really helped tackle some of these bugs. If I could also look at the code at certain points when pausing it would have been even better. Some mistakes were probably due to sloppiness on my part, the old "if it type checks, it works" saying at the back of my head might have made me less careful on my variable naming and such. Maybe that infinite recursion could have been avoided.

Second set of problems: working with Elm was not without problems.

Only syntax highlighting plugin for Vim was available. I needed to run the game through elm-reactor to see parsing/typing/etc bugs.
At times Elm felt still an immature platform to work on. Many parsing bugs were cryptic, not so informative and were sometimes not placed close to the actual error.
I was not able to hot-swap, restart, or unpaused with Elm-reactor.
There was a problem that when trying to compile, elm-make would just throw an "out of memory" error. I really don't know how I solved that
Elm 0.15 was announced a few hours before I finished and a package I used no longer appeared on package.elm-lang.org so I had to look at the code for documentation.

Things I didn't have enough time to do:

Animations
Integrating audio (I could no longer find the elm-audio package on package.elm-lang.org after the announcement to 0.15) though I recorded some music and sfx.
Fixing the maze generator -> many levels
Score, lives and such
Screens

Conclusion

I had fun, I learned a lot, and I was really happy to finally use functional programming for games. Even though I got stuck a lot of times, have had to chase and solve many bugs due to faulty logic and didn't really finish the game, It was a very eye-opening experience and I felt like I gained a lot from it.

Elm is still a little rough around the edges and so it is not so easy to work with, at the moment, during a 72 hour game jam, from scratch, with minimal prior knowledge on it, But it is also very young. I have a feeling that if it keeps in that direction - work hard on better tools like elm-reactor, work on the eco-system and packages, fix bugs and such, for another 3 years, it will become an incredible programming language and system to work with.

The next time I participate on Ludum Dare, it will most likely be with Elm. This time with better understanding of it and hopefully, a much better result :)

Thanks for reading!

I Have Never…

huphtur — Tue, 21 Apr 2015 03:00:00 GMT

Done drugs
Drank coffee
Worn a scarf
Been camping
Watched Titanic
Smoked cigarettes
Listened to a Drake song (voluntarily)
Believed in a higher power (besides my watts)

…and most likely never will.

Silicon Valley: Season 2

🦉 olshansky 🦁 — Mon, 20 Apr 2015 16:09:33 GMT

Olshansky's review of Silicon Valley: Season 2

Game of Thrones: Season 5

🦉 olshansky 🦁 — Mon, 20 Apr 2015 16:01:29 GMT

Olshansky's review of Game of Thrones: Season 5

Docker containers in Oh-My-Vagrant

The Technical Blog of James on purpleidea.com — Mon, 20 Apr 2015 07:35:43 GMT

The Oh-My-Vagrant (omv) project is an easy way to bootstrap a development environment. It is particularly useful for spinning up an arbitrary number of virtual machines in Vagrant without writing ruby code. For multi-machine container development, omv can be used to help this happen more naturally.

Oh-My-Vagrant can be very useful as a docker application development environment. I’ve made a quick (<9min) screencast demoing this topic. Please have a look:

https://dl.fedoraproject.org/pub/alt/purpleidea/screencasts/oh-my-vagrant-docker-screencast.ogv

F# Vs C#

Nicola Iarocci — Mon, 20 Apr 2015 03:00:00 GMT

I have another guest post up at the official MSDN Team Blog. Titled F# Versus C#, it is an attempt at a gentle introduction to F# for the Italian C# developer.

Of type lists and type switches

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Sun, 19 Apr 2015 16:03:24 GMT

Every C++ programmer sooner or later happens to stumble over template meta-programming. I was no exception. And suddenly, the old adage applied to me as well: "If all you have is a hammer, everything starts to look like a nail"

A very useful technique from template meta-programming is the concept of type lists. Briefly put, a type list is a list that contains types instead of values. It is thus defined at compile-time already. When might this be useful? In itself, probably never. Except for showing off. But in my opinion, the real utility of type lists unfolds when they are coupled with type switches. A type switch is another—you guessed it—concept from meta-programming. Given a type list, a type switch offers a way for calling a functor with a given type from the type list. The whole point is that the type switch permits the selection of a functor at runtime! Hold on to your hats, gentlefolks! This means I can essentially write something like this:

typedef TypeList<char, short, int, long> IntegerTypes;
Functor f;
TypeSwitch<IntegerTypes> ts;

unsigned int index = 0;
std::cin >> index;

ts( index, f );

The snippet above does not compile yet, but we’ll get there! You can see that we have deferred the decision about which type to use in the functor call. We let a user input at runtime decide about the type. Even though we are using templates. When I first learned about this, it seemed like magic to me. As we all know, any sufficiently analysed magic is indistinguishable from science, though. So, let’s try to get there. Throughout this post, I will include some snippets from real code. You can get the complete project on GitHub.

Type list

There are many ways for designing a type list. I chose a very easy one by limiting my type list to 10 types, which spared me from having to define it recursively. Andrei Alexandrescu has a great article about recursively-defined type lists and their applications. We start out differently, by defining a generic base case first:

struct EmptyType
{
};

template <
    typename T0 = EmptyType,
    typename T1 = EmptyType,
    typename T2 = EmptyType,
    typename T3 = EmptyType,
    typename T4 = EmptyType,
    typename T5 = EmptyType,
    typename T6 = EmptyType,
    typename T7 = EmptyType,
    typename T8 = EmptyType,
    typename T9 = EmptyType
> struct TypeList;

So far, nothing wild is going on. We have an empty type and a template declaration. We shall now use template specialization to refine the type list:

template <
    typename T0,
    typename T1,
    typename T2,
    typename T3,
    typename T4,
    typename T5,
    typename T6,
    typename T7,
    typename T8,
    typename T9
> struct TypeList
{
  typedef T0 head;
  typedef TypeList<T1,T2,T3,T4,T5,T6,T7,T8,T9> tail;

  enum
  {
    length = tail::length + 1
  };
};

This is also not very spectacular. We have a typedef for the head and a reduced type for the tail of the list. We also define a recursive enum. This is allowed because tail refers to a type that is known at compile-time. We are not finished, though. We still need to define a base case, namely that of the fully empty type list:

template <> struct TypeList<
    EmptyType,
    EmptyType,
    EmptyType,
    EmptyType,
    EmptyType,
    EmptyType,
    EmptyType,
    EmptyType,
    EmptyType,
    EmptyType>
{
  enum
  {
    length = 0
  };
};

And there we have it! A simple type list that is utterly useless. Well, we can at least query its size:

typedef TypeList<char, short, int, long> IntegerTypes;
std::cout << IntegerTypes::length << std::endl;

This should yield an output of 4. We can capture this in a nice struct:

template <typename TypeList> struct TypeListLength
{
  enum
  {
    value = TypeList::length
  };
};

This permits us to write TypeListLength<IntegerTypes>::value, making the intent somewhat clearer. Also, it makes writing an access struct easier.

Accessing a type from a type list

A useful operation for the type list would be an access functor. Given an index, we should be able to query the time at that index. If the index is out of range, we refuse to compile. That’s at least one good thing about templates—we catch problems early with very confusing compiler output. Let’s start with an access functor then:

template <
  typename TypeList,
  unsigned int Index,
  unsigned int Step = 0,
  bool Stop = ( Index == Step ),
  bool OutOfRange = ( TypeListLength<TypeList>::value == 0 )
> struct TypeListGet
{
  typedef typename TypeListGet<typename TypeList::tail, Index, Step + 1>::type type;
};

This is only the general case. We have an Index attribute that refers to the desired index in the type list by the client, and some "internal" attributes for keeping track of where we are in the type list.

Now let’s handle the specialization. The first one is easy. If we have reached the end of the type list, we define the struct to be empty. This will result in a compile-time error when a client aims to access an incorrect index:

template <
  typename TypeList,
  unsigned int Index,
  unsigned int Step,
  bool Stop
> struct TypeListGet<
  TypeList,
  Index,
  Step,
  Stop,
  true>
{
  // This is empty by design.
};

The other specialization is straightforward as well. In case Index and Step coincide, the Stop attribute is set to true. This means we have found our desired type and it is at the very beginning of the type list. Hence:

template <
  typename TypeList,
  unsigned int Index,
  unsigned int Step,
  bool OutOfRange
> struct TypeListGet<
  TypeList,
  Index,
  Step,
  true,
  OutOfRange>
{
  typedef typename TypeList::head type;
};

Now our demo program may become slightly more involved:

#include "type_list.hh"

#include <iostream>
#include <typeinfo>

int main( int, char** )
{
  typedef TypeList<char, short, int, long> IntegerTypeList;

  std::cout << IntegerTypeList::length << "\n";

  std::cout << "The type list has a length of " << TypeListLength<IntegerTypeList>::value << std::endl
            << "The first type is " << typeid( TypeListGet<IntegerTypeList, 0>::type ).name() << std::endl
            << "The second type is " << typeid( TypeListGet<IntegerTypeList, 1>::type ).name() << std::endl
            << "The third type is " << typeid( TypeListGet<IntegerTypeList, 2>::type ).name() << std::endl
            << "The fourth type is " << typeid( TypeListGet<IntegerTypeList, 3>::type ).name() << std::endl;

  return 0;
}

The output should read like this:

The type list has a length of 4
The first type is c
The second type is s
The third type is i
The fourth type is l

Still not terribly useful, but somewhat cool, I guess.

The type switch

Now let’s add the magic. Given a functor that provides a templated operator()(), we want to call it with a type that is selected at runtime using an index. The generalized case is very ugly here because we do not know the return type of the functor beforehand. Lucky for us, C++11 offers the decltype specifier:

template <
  typename TypeList,
  unsigned int Index = 0,
  bool Stop = ( Index == TypeListLength<TypeList>::value )
> struct TypeSwitch
{
  template <class Functor> decltype( Functor().template operator()<typename TypeListGet<TypeList, Index>::type>() ) operator()( unsigned int i, Functor f )
  {
    if( i == Index )
      return f.template operator()<typename TypeListGet<TypeList, Index>::type>();
    else
    {
      TypeSwitch<TypeList, Index + 1> next;
      return next.operator()( i, f );
    }
  }
};

The decltype ensures that we get the proper return type of the functor for the current type at the requested index of the type list. If we have not found the proper index yet, we create a new type switch (with adjusted index) and recurse.

Now the only thing that is left to handle is the when the index goes out of bounds. Here, we can use a specialization with a fixed value for the Stop attribute. Recall that it is set when the index is equal to the length of the type list, i.e. upon encountering the first invalid index.

template <
  typename TypeList,
  unsigned int Index
> struct TypeSwitch<
  TypeList,
  Index,
  true>
{
  template <class Functor> decltype( Functor().template operator()<EmptyType>() ) operator()( unsigned int /* i */, Functor /* f */ )
  {
    throw std::runtime_error( "Index is out of range" );
  }
};

And that’s it. We may now use such cool constructions as the following:

#include "type_list.hh"
#include "type_switch.hh"

#include <iostream>
#include <typeinfo>

struct Functor
{
  template <class T> void operator()()
  {
    std::cout << "Called with the following type: " << typeid(T).name() << std::endl;
  }
};

int main( int, char** )
{
  typedef TypeList<char, bool> MyTypes;

  Functor f;
  TypeSwitch<MyTypes> ts;

  for( unsigned int i = 0; i < TypeListLength<MyTypes>::value; i++ )
    ts( i, f );

  try
  {
    ts( 42, f );
  }
  catch( std::runtime_error& e )
  {
    std::cerr << "Oops, this type does not exist: " << e.what() << std::endl;
  }

  return 0;
}

This results in the following output:

Called with the following type: c
Called with the following type: b
Oops, this type does not exist: Index is out of range

OK, maybe the program is not completely useful. In general, type switches can be very useful, though. I have used it in the past to loop over pre-defined types for volume data. The functor in this case simply checked whether it was possible to load volume data of a fixed size with a given type. If so, we knew that we had identified the right type. The advantage of this approach is that it naturally scales with the number of types we define. We do not have to resort to ugly dynamic_cast constructions or something like that.

If you want to learn more, I can only recommend the books by Andrei Alexandrescu. If you are a bit like me, please also take JWZ’s law into account:

Some people, when confronted with a problem, think "I know, I’ll use ~~regular expressions~~ template meta-programming." Now they have two problems.

Enjoy your templates!

I want the code

Sure, you can have it. It’s on GitHub and licensed under an MIT license.

Tokyo travel bookmarks

xenodium.com @alvaro — Sun, 19 Apr 2015 03:00:00 GMT

14 Things to Know Before You Go to Tokyo.
5 alternative things to do in Tokyo (by globalhelpswap).
A History of Tokyo in 8 Dishes.
Akasaka area (N/E of Roppongi).
Akasaka Sagamiya (also Mamekan, established in 1895).
Akihabara: Electronics district, arcades and comic stores.
Asakusa (tourist spot). Kaminarimon, Nakamise (oldest shopping street in Japan). Sensoji Temple.
Asakusa hotel.
Asakusa Shrine: Shinto shrine and market.
Bar Epilogue.
Bar Odin.
Benten, Taito - 3-21-8 Asakusa, Ueno, Asakusa (Buckwheat noodles shop).
Bentomi (Sushi).
CHEESE STAND （チーズスタンド）.
Coffee Tengoku, Asakusa (best hotcakes/pancakes). Noting is premade. After 12 years, best hotcakes.
Cup noodle museum.
Daiwasushi (food). An account here.
Dorayaki (in Ukenbukuru). Seijuken is long established in Ningyocho.
Doteno-Iseya 土手の伊勢屋: 127-year old Tempura.
Eateries around Takadanobaba eki
Ebisu (area for izakaya experience, drink/chat with bartender). Look for Ebisu Yokocho.
Fish market (6am sushi).
Five of the worst areas to live in and around Tokyo | SoraNews24.
Fu-unji & 風雲児 | Tokyo Eats.
Ghibli museum (book in advance or try lawson).
Ginza Kimuraya (bakery). Try anpan (invented here).
Ginza.
Ginzakyuubee (food).
Golden Gai (lots tiny bars).
How to Spend a Magical Christmas in Tokyo (2019 update).
https://www.timeout.com/tokyo/restaurants/tokyo-coolest-kakigori
Ichiran Ramen.
Imperial Palace: Book to go inside. Beautiful park, great for pictures.
In which ward are you living?.
Irie known (sweet shop) for Mamekan. In Monzen-Nakacho.
Isetan (upscale food market).
Isozushi (food).
Jimbocho Book Town (largest second-hand book town in the world, classic-style cafes, old school charm).
Kaiden-don, near tsukiji (Sushi).
Kajitsuen Libre. Peach Parfait. Fresh fruit.
Kinozen restaurant for Matcha Bavarian cream (matcha bavaroa). Kagurazaka. Check out the backstreets.
Kodai's Fukui restaurant (old tea house).
Kushikatsu restaurant in Golden Gai, a karaoke booth in Ebisu, or drifting through the back alley izakayas of Akabane.
Menya Musashi and Fuunji are both generally very well regarded and about a 5-7 minute walk from the SW part of Shinjuku station.
Mikimoto building in Ginza.
Mori Art museum.
Musashiya ramen.
Nakiryu: Tantalizing Tantanmen Noodles with a Michelin Star.
Nezu museum.
Odaiba.
Okonomiyaki in Tokyo.
Omoide yokocho: Alleyway next to Shinjuku station. Lots of yakitori restaurants.
Omotesando Koffee (coffee and baked custard slice).
Omotesando side streets.
Pan No Tora Bakery (Kyoto?).
Pignon | French Restaurant.
Pound cake from Patisserie Gondola in Kudanue.
Roppongi.
Savarin (brioge-based, of french origin) in Yokohama. Cafe Recherche.
Seafood street south of Ueno eki
Secret snapshots of Tokyo vivid street life.
Shibamata: Snacking and Sightseeing in Tokyo’s Old Edo Neighborhood.
Shibuya Crossing: Largest pedestrian crossing in the world.
Shimokitazawa (Shimokita for short). Bohemian, vintage shopping.
Shimokitazawa (thirft stores, music bands, pubs, and cafes).
Short trips from Tokyo.
Short trips from Tokyo.
Sukiyabashi Jiro.
Sushi dai (Sushi, long queue, maybe turisty).
Sushi Kanesaka.
Sushi places around Tsukiji market in the morning (much better than the market visit itself, these days)
Sushi Saito.
Sushidai (food).
Sushitsu (food).
Sutekihausukatsura.
Takaosan (Mount Takao).
Tempura Tsunahachi en Tokio: 4 opiniones y 7 fotos.
Tenkazushi (food).
The whole of Kabukicho (area in-between Shinjuku station and Shin-Okubo)
Things to Do in Tokyo Japan - Sunday Spotlight.
Tokyo Gov on Twitter: Higashikurume Station.
Tokyo in winter: what to see, do and eat (Lonely Planet).
Tokyo Municipal Government building: Only for observation deck with view to Fuji (if clear day).
Tokyo on a budget: tips for making your yen go further (Lonely Planet).
Tokyo play blog.
Tokyo Salaries: all you need to know.
Tokyo station: Friendly JR office (english spoken). They help book all trips/tickets/reservations.
Tokyo Station: Massive station. Lots of restaurants and shops (check out ramen street).
Tokyo Travel Tips: 5 Things You Need In Japan | Coffee and Passport.
Tokyo Twitter: any Japanese breakfast restaurants.
Tokyo, Japan - Condé Nast Traveller | CN Traveller.
TokyoCheapo.
Tokyo’s oldest train line – in pictures | Art and design | The Guardian.
Tokyu Hands and Loft (shops in Shibuya).
Toritake (yakitori at Shibuya).
Try Mos rice burger (fast food).
Try out Pancan (bread in can).
Tsukiji Fish Market calendar.
Tsukiji Fish Market: Sushi bars and food vendors (get there early, visitor numbers restricted).
Umemura in Asakusa (also Mamekan).
Unusual things to do in Tokyo? .
Walk around Yoyogi-Uehara Station, Shibuya, Binaural City Sounds(ASMR) in Tok….
What's something you know about in Tokyo that you think others should know about?.
Yurakucho Yakitori Alley
Zauo.com "Let's fixhing enjoy". Catch your own fish to eat.
婁熊東京 (raw and grilled pork).
酒友 (Sake & good Shabushabu), Roppongi.

UK travel bookmarks

xenodium.com @alvaro — Sun, 19 Apr 2015 03:00:00 GMT

Development quotes

xenodium.com @alvaro — Sun, 19 Apr 2015 03:00:00 GMT

Development philosophy

xenodium.com @alvaro — Sun, 19 Apr 2015 03:00:00 GMT

Boyscout rule: Leave campground cleaner than found.
What can clang-format teach us about the human condition?.

Spain travel bookmarks

xenodium.com @alvaro — Sun, 19 Apr 2015 03:00:00 GMT

Meet up bookmarks

xenodium.com @alvaro — Sun, 19 Apr 2015 03:00:00 GMT

2015-04-19

ho.dges.online — Sun, 19 Apr 2015 03:00:00 GMT

Probably my favourite ever picture of B and one of the very few occasions when I managed to harness the magic of the Fuji 56mm lens.

Every app is a superpower

Steph Ango — Sun, 19 Apr 2015 03:00:00 GMT

When Google Maps first came out, it blew me away. It still seems like one of the most magical technologies we have today.

I wonder what Ferdinand Magellan would think if you showed him Google Maps on a phone? The entire world mapped in detail, with photos of every building in every city.

I sat in an airport listening to Pyotr Ilyich Tchaikovsky on Spotify, wondering what he’d think if he knew that anyone can listen to high-fidelity recordings of all his compositions. Just a phone and a pair of earbuds — no matter where you are.

What if Johannes Gutenberg could hold the world’s knowledge in the palm of his hands, and search Wikipedia for anything he can think of?

Imagine showing Nicéphore Niépce and the Lumière brothers what images they could capture from a camera that fits in your pocket.

I would love to see Alexander Graham Bell wirelessly have a face-to-face conversation with anyone on Earth.

Within ten seconds, I can tell you what the weather will be like tomorrow or what the top five restaurants are, for any city in the world. I can have a chauffeur here in 5 minutes, or have my groceries delivered before dinner.

In our pockets is a portal to powers that even kings couldn’t dream of.

Plantuml example

xenodium.com @alvaro — Fri, 17 Apr 2015 03:00:00 GMT

Played with Plantuml. Convenient for generating UML diagrams from text. Here's the Language Reference Guide. Here's an example:

@startuml
  abstract class Singer {
    abstract void sing()
    void Dance()
  }

  skinparam monochrome true
  Singer <|-- PopSinger
  Singer <|-- SalsaSinger

  class PopSinger {
    void sing()
  }

  class SalsaSinger {
    void sing()
  }

@enduml

Install plantuml on Mac OS X:

brew install plantum

Generating diagram:

$GRAPHVIZ_DOT=~/homebrew/bin/dot java -jar path/to/plantuml.8018.jar diagram.plantuml

ps. Installation and verification gist.

ps2. More handy UML examples in this fork.

Fattura Elettronica Open Source Update

Nicola Iarocci — Fri, 17 Apr 2015 03:00:00 GMT

Fattura Elettronica Open Source has been updated to v0.1.3 a few days ago and is available on NuGet. Sources are on GitHub. It fixes a deserialization issue with the ReadXML method.

A cost-based scheduler for ETL pipelines

Tafkas Blog — Fri, 17 Apr 2015 03:00:00 GMT

To speed up the ETL data pipeline, you should try to run jobs in parallel. Obviously, not all jobs can run at the same time in most cases, since there are dependency constraints between the jobs and limits of the servers capacity (number of processors and/or IO bandwidth). So assuming the server allows you to run n jobs in parallel, often there is the situation that the dependencies give you the option to run any of a set of m different jobs with m > n.

Go from a PHP Perspective

Brightball Articles — Fri, 17 Apr 2015 00:06:59 GMT

Here are the slides from my recent presentation to UpstatePHP in Greenville, looking at Go (Golang) from a PHP Perspective.

The Gambler

🦉 olshansky 🦁 — Thu, 16 Apr 2015 18:52:27 GMT

Olshansky's review of The Gambler

Anvil: big progress for a small library

zserge's blog — Thu, 16 Apr 2015 03:00:00 GMT

Anvil: big progress for a small library I have just released Anvil 0.0.2. So many changes have been made and now I am absolutely confident that Anvil has its strong points even though I’m the only active user of it. What is Anvil? This section is for those who missed the early steps of Anvil. Anvil is a tiny reactive UI library for Android. What does this mean? It means that you can declare your UI in Java, bind your data to the views, bind event listeners to the views and leave it running.

Where Anvil meets Kotlin

zserge's blog — Thu, 16 Apr 2015 03:00:00 GMT

Kotlin is a very nice language for JVM. I wrote about it in the past. Anvil is a very nice UI library for Android. How do these two play together? The first attempt My favourite example to demonstrate Anvil syntax is a simple click counter. It demonstrates the use of layouts, views, data binding and event binding. In Java it looks like this: public ViewNode view() { return v(LinearLayout.class, orientation(LinearLayout.VERTICAL), v(TextView.

Taken 3

🦉 olshansky 🦁 — Wed, 15 Apr 2015 17:45:39 GMT

Olshansky's review of Taken 3

Star Wars: Episode IV - A New Hope

🦉 olshansky 🦁 — Wed, 15 Apr 2015 17:39:27 GMT

Olshansky's review of Star Wars: Episode IV - A New Hope

Numpy matrices work for general objects

Negative Feedback — Wed, 15 Apr 2015 11:12:54 GMT

A student was having difficulty using a class I had created to represent transfer function objects. I discovered that they were actually placing these objects in a numpy.matrix and trying to multiply things by one another. To my surprise, this actually worked! I had to investigate. The code here is all in Python.

I created a quick tracing object which just kind of logs the operations that are done on it as an expression:

class operation:
    def __init__(self, name):
        self.string = name
    def __repr__(self):
        return self.string
    def __str__(self):
        return self.string
    def __add__(self, other):
        return operation("({} + {})".format(str(self), str(other)))
    def __mul__(self, other):
        return operation("{}*{}".format(str(self), str(other)))
    def __radd__(self, other):
        return operation("({} + {})".format(str(other), str(self)))
    def __rmul__(self, other):
        return operation("{}*{}".format(str(other), str(self)))

Now I can do some interesting stuff:

    In []: a*b
    Out[]: a*b
    In []: a + b
    Out[]: (a + b)

Ok, but what about matrix multiplication?

import numpy as np
In []: tracer = np.matrix([[a, b, c],
[d, e, f],
[g, h, i]])

In []: tracer*tracer
Out[]:
matrix([[((a*a + b*d) + c*g), ((a*b + b*e) + c*h), ((a*c + b*f) + c*i)],
        [((d*a + e*d) + f*g), ((d*b + e*e) + f*h), ((d*c + e*f) + f*i)],
        [((g*a + h*d) + i*g), ((g*b + h*e) + i*h), ((g*c + h*f) + i*i)]], dtype=object)

That kind of blew me away! It means I can probably get away with a whole lot less code than I thought I would need to implement my own matrix-like objects.

In addition, it appears that some numpy functions will attempt to call corresponding functions in the object they act on:

    In []: np.exp(a)
---------------------------------------------------------------------------

AttributeError                            Traceback (most recent call last)
 in ()
----> 1 np.exp(a)
AttributeError: exp

This error shows that np.exp is trying to access the exp attribute on this object. Let's extend our object with an exp method like this:

    def exp(self):
        return operation("e^({})".format(self))

Now (after re-instantiating a-i from above), we have

In []: np.exp(a)
Out[]: e^(a)

This is going to be super useful.

Burton Snowboards, the Anti-Apple

JonPaulUritis.com — Wed, 15 Apr 2015 03:00:00 GMT

One of the most disturbing aspects of design trends over the last decade or two is the cult of Apple, and the various sound bytes that they use to be able to convert people into their design aesthetic => my specific issue is with this hyperbolic “minimalism” mantra that they have been spoon feeding Americans for the better part of 20 years now.

Now its important to note I am not against “minimalism”, “simplification”, “usability”, or any of the other things that Apple likes to focus on with their products and marketing. In fact, I actually see them as very important design aesthetics and mantras. What I am against is the notion that minimalism is the sole design aesthetic, and that it is somehow the greater good thus a reason to build a dogma around it.

Minimalism should be in context. Where would we be without Picasso’s cubism or Cézanne beautiful use of color or Van Gogh’s deep detail that required careful consideration. It was only later in Pop-Art and Minimalism movements that we felt the need to reduce twice over as an “antithesis” for many of the busy and colorful artistic movements from before.

Thankfully, when I was reading Purple Cow it Came to me that one of my all time favorite companies, Burton Snowboards is actually an exact opposite to Apple inc. Here are some examples:

Major Features of Apple:

Minimalism
Plain high contrast colors
“easy to use”
User friendly
Design over Technology ( consistently a technology laggard technically by specifications)
Very few products
Larger than life CEO

Major Features of Burton:

Active Designs
Products that are harder to use => Snowboarding over skiing
Extremely colorful
Technology leads to design
Many products and varying features to each product line
CEO that prefers to be out snowboarding

Burton’s approach is pretty much the exact opposite to Apple but their brand and products are every bit as strong as Apples (they just function in a smaller marketplace). For Example, Burton is aggressive about creating the best technology where as Apple traditionally sells technology that is 20% slower, less powerful, and smaller in capacity than the existing marketplace. Burton is very “active” with their designs and they consistently out design their competition (competition that focuses on Minimalism). Additionally, they have tons of products- yes they primarily sell snowboards but you have many choices within the Burton environment.

I think there is a lot to what Burton is doing that other Companies could learn from, and I would urge people to consider their design Aesthetic in context.

Apple is great, but that doesn’t mean they are the only ones that can get it right.

Unicode and Matlab on the command line

bastibe.de — Wed, 15 Apr 2015 03:00:00 GMT

As per the latest Stackoverflow Developer Survey, Matlab is one of the most dreaded tools out there. I run into Matlab-related trouble daily. In all honesty, I have never seen a programming language as user-hostile and as badly designed as this.

So here is today's problem: When run from the command line, Matlab does not render unicode characters (on OSX).

I say "(on OSX)", because on Windows, it does not print a damn thing. Nope, no disp output for Windows users.

More analysis: It's not that Matlab does not render unicode characters at all when run from the command line. Instead, it renders them as 0x1a aka SUB aka substitute character. In other words, it tries to render unicode as ASCII (which doesn't work), and then replaces all non-ASCII characters with SUB. This is actually reasonable if Matlab were running on a machine that can't handle unicode. This is not a correct assessment of post-90s Macs, though.

To see why Matlab would do such a dastardly deed, you can use feature('locale') to get information about the encoding Matlab uses. On Windows and OS X, this defaults to either ISO-8859-1 (when your locale is pure de_DE or en_US) or US-ASCII, if it is something impure. In my case, German dates but English text. Because US-ASCII is obviously the most all-encompassing choice for such mixed-languages environments.

But luckily, there is help. Matlab has a widely documented (not) and easily discoverable (not) configuration option to change this: To change Matlab's encoding settings, edit %MATLABROOT%/bin/lcdata.xml, and look for the entry for your locale. For me, this is one of

<locale name="de_DE" encoding="ISO-8859-1" xpg_name="de_DE.ISO8859-1"> ...
<locale name="en_US" encoding="ISO-8859-1" xpg_name="en_US.ISO8859-1"> ...

In order to make Matlab's encoding default to UTF-8, change the entry for your locale to

<locale name="de_DE" encoding="UTF-8" xpg_name="de_DE.UTF-8"> ...
<locale name="en_US" encoding="UTF-8" xpg_name="en_US.UTF-8"> ...

With that, Matlab will print UTF-8 to the terminal.

You still can't type unicode characters to the command prompt, of course. But who would want that anyway, I dare ask. Of course, what with Matlab being basically free, and frequently updated, we can forgive such foibles easily...

Talk - High Performance Software Defined Storage

smcleod.net — Wed, 15 Apr 2015 03:00:00 GMT

A high level talk from Infracoders Melbourne on 12/04/2015.

There’s also a low quality recording available here:

Building a high performance SSD SAN - Part 1

Filetree Listing

Ben E. C. Boyter — Wed, 15 Apr 2015 00:32:31 GMT

Just a quick update to searchcode. A few small tweaks here and there, but the largest is that there is now a file tree listing option which will show the file tree for any project. An example would be going to this file and then clicking the “View File Tree” button on the top left.

An example screenshot of the result of this is included below.

Helm-describe-helm-attribute

xenodium.com @alvaro — Tue, 14 Apr 2015 03:00:00 GMT

Writing A Spotify Client in 16 Minutes is fantastic for picking up helm and Emacs lisp tips. Of interest helm-describe-helm-attribute, second to the awesomeness of helm-spotify integration.

<iframe width='420'
        height='315'
        src='https://www.youtube.com/embed/XjKtkEMUYGc'
        frameborder='0'
        allowfullscreen>
</iframe>

Forget Not His Benefits

trv.rs — Tue, 14 Apr 2015 03:00:00 GMT

Bless the Lord, O my soul, and forget not all his benefits, who forgives all your iniquity, who heals all your diseases, who redeems your life from the pit, who crowns you with steadfast love and mercy, (‭Psalm‬ ‭103‬:‭2-4‬ ESV)

This whole passage is glorious, but I've been reflecting on being crowned with steadfast love and mercy... God is a good father, and it seems to me that He would not "crown" us unless He had already provided for all our basic needs. Washed clean by His blood, fed by His Word, clothed in His righteousness, crowned by His faithfulness and mercy.

I'm convicted that I have often forgotten this benefit, indeed I even enter prayer without acknowledging any of these glorious truths... Asking for so much, I end up praising for so little. I'm going to spend some serious time this week revisiting "his benefits." It's already been a sweet thing for my soul. Thank you Lord!

Poem by Simon Armitage

ho.dges.online — Tue, 14 Apr 2015 03:00:00 GMT

And if it snowed and snow covered the drive
he took a spade and tossed it to one side.
And always tucked his daughter up at night
And slippered her the one time that she lied.
And every week he tipped up half his wage.
And what he didn’t spend each week he saved.
And praised his wife for every meal she made.
And once, for laughing, punched her in the face.

The Sea of Tranquility

Apurva Shukla's RSS Feed — Mon, 13 Apr 2015 10:00:00 GMT

⭐ ⭐ ⭐ ⭐ ⭐ I started reading this book based on a friend’s recommendation and a few chapters in I started to regret it. For me, the…

Tips for Building Mobile Games in HTML5

Thomas Hunter II — Mon, 13 Apr 2015 03:00:00 GMT

How To Automatically Update Your Angular Offline Webapps

Nicolas Goutay — Mon, 13 Apr 2015 03:00:00 GMT

Offline-first webapps are a gracious way to guarantee a worst-case user experience that can be controlled and fine-tuned by developers.

Explaining the need for privacy

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Sun, 12 Apr 2015 15:48:27 GMT

John Oliver recently explained the need for privacy in his great show Last Week Tonight by using a rather NSFW example, namely his private parts. There is even a new website about this very issue. I love the internet.

However, I am not entirely satisfied with the reason for privacy. Taking and sending pictures of your junk is a deliberate action of your part. Eric Schmidt had the following to say about this:

If you have something that you don’t want anyone to know, maybe you shouldn’t be doing it in the first place.

This is certainly not completely wrong—and this is precisely the reason why I dislike the example used by John Oliver. Let me explain: In German, we have two words for privacy. We have Intimsphäre, which pertains to things that are “intimate”, e.g. your junk. We also have the word Privatsphäre, which pertains to things that are (merely) private. In my opinion, it is a bad idea to reduce the problem with surveillance to the most intimate of things! Of course, most people would not like the government to make copies of their nude pictures or whatever. That is beside the point. I even resent governments or anyone snooping in things that are clearly private.

A more useful example is the current state of my apartment. When I am having people over, I clean everything to make sure that no discarded clothes, papers, or anything messy is around. When I am on my own, books start to occupy all surfaces over time. This is not something I need to hide, but I would still be embarassed if I am having a visitor and the apartment is not cleaned. Now imagine the government being able to look over your daily affairs, with all the social make-up removed. All your journal entries, e-mails to loved ones (with or without pictures of private parts), comments on social networks, and so on. This is the real issue with (government) surveillance. We must not let the NSA and others take over our Privatsphäre and fall back to our Intimsphäre. They need to be kept out of both!

(I wrote this as a result of a very intriguing coffee-break discussion with friends and colleagues. Thank you for the insights)

Youtube videos in your org html export

xenodium.com @alvaro — Sun, 12 Apr 2015 03:00:00 GMT

Sacha Chua and John Wiegley posted a wonderful video on Emacs lisp development tips. Embedding the following raw HTML using #+BEGIN_HTML/#+END_HTML:

<iframe width="420"
        height="315"
        src="https://www.youtube.com/embed/QRBcm6jFJ3Q"
        frameborder="0"
        allowfullscreen>
</iframe>

results in an embedded video when exporting your org file:

<iframe width="420"
        height="315"
        src="https://www.youtube.com/embed/QRBcm6jFJ3Q"
        frameborder="0"
        allowfullscreen>
</iframe>

.net bookmarks

xenodium.com @alvaro — Sun, 12 Apr 2015 03:00:00 GMT

Open Source .Net libraries that make your life easier.

Decisions in Pillars of Eternity

bastibe.de — Sun, 12 Apr 2015 03:00:00 GMT

Early on in Pillars of Eternity, you are tasked to kill one of two characters: Either you kill King Rethoric, who executed many innocent people, or you kill Rolsc, the leader of the rebellion. This is an interesting moral choice, but it is also profoundly sad that the game presents killing either of them as the only resolution to this conflict.

It got me thinking. In the real world, I would not consider killing to be an option, ever. But in the game, you have to play by the game's rules. And furthermore, you can't just walk away and have the two characters duke it out amongst themselves: The whole game is built around the player, and without the player's interaction, the game world doesn't evolve.

Thus, this conflict won't resolve itself, and the player is forced to kill. I wish there were a diplomatic option, or a way of fixing the underlying problem so the two characters are not at odds any more. I fear how these choices in video games might influence our perceptions of everyday choices, and crave for video games that offer interesting choices that do not revolve around murder.

From what I hear, Planescape Torment did offer such choices. As did many of my favorite games of all times.

HP Support Solutions Framework Security Issue

Tom Forbes — Sun, 12 Apr 2015 02:50:23 GMT

After discovering the flaw in Dell’s System Detect software I looked into other similar software for issues. This post details two issues I found with the HP Product Detection software and explores the protections HP put in place. I’m also going to explain how they could be easily bypassed to allow ...

A Brief History of Time

🦉 olshansky 🦁 — Sat, 11 Apr 2015 03:00:00 GMT

Olshansky's review of A Brief History of Time by Stephen Hawking

UK property bookmarks

xenodium.com @alvaro — Fri, 10 Apr 2015 03:00:00 GMT

Commute from (find property based on potential commute).
Mapumental Property (find property by travel time).

Git commit message style

xenodium.com @alvaro — Thu, 09 Apr 2015 03:00:00 GMT

Adopted Tim Pope's Git commit message style. Also enabled Emacs's git-commit-training-wheels-mode:

(use-package git-commit-training-wheels-mode :ensure t
  :commands (git-commit-training-wheels-mode))

(use-package git-commit-mode :ensure t
  :config
  (add-hook 'git-commit-mode-hook 'git-commit-training-wheels-mode)
  :commands (git-commit-mode))

Another great post by Chris Beams.

Sharing dev environments with Oh-My-Vagrant

The Technical Blog of James on purpleidea.com — Wed, 08 Apr 2015 06:37:18 GMT

With Oh-My-Vagrant (omv) you can set up a dev environment in seconds. (Read the omv introduction if you’ve never used it before!) Since everything is defined in a single omv.yaml file, it is easy to share your cluster prototype with a friend! The one missing feature was associating code with this config file. This is now possible! Let me show you how it works…

In the omv.yaml file there is an extern variable. It is a list of each external repository which you’d like to include. Each element in this list is a hash of key value pairs. Currently four are supported: type, system, repository, directory.

"many more gadgets will surely be built with FPGAs"

Welcome to Marco Fioretti's website! on Stop at Zona-M — Tue, 07 Apr 2015 09:10:00 GMT

Three and a half years ago, I explained why it’s time to bring FPGAs to the masses. One year later, I wrote that this is the time for European microprocessors and FPGAs.. And guess what I just read today?

Coursera: Principles of Reactive Programming

GeekWare - Daniel Pecos Martínez — Tue, 07 Apr 2015 05:32:33 GMT

In little less than a week, a new edition of Coursera’s course, Principles of Reactive Programming from Martin Odersky, Erik Meijer and Roland Kuhn, is launching.

This course is a follow-up of Principles of Functional Programming in Scala, which I took in its first edition and which I enjoyed a lot. I’ve regretted missing out first edition of this second course for a long time (more than a year, in fact), so if you want to learn about this magnificent frameworks of the Scala world, don’t miss this chance!

And best of all, it’s free!

So Long and Thanks for All the Job Jackets

Sam Solomon — Tue, 07 Apr 2015 03:00:00 GMT

April 1, 2013 was my first day working at Red Square Agency as an interactive producer. April 1, 2015 was my first day working full-time on SidePrize.

Ruby on Rails and PostgreSQL Class Slides

Brightball Articles — Mon, 06 Apr 2015 19:03:00 GMT

In August I taught a course titled Ruby on Rails and PostgreSQL - Intro to Advanced in Greenville over the span of 3 weeks. Here is the compilation of slides from the class.

fci-mode and org-html-export-to-html bug

xenodium.com @alvaro — Mon, 06 Apr 2015 03:00:00 GMT

Having enabled fci-mode in most programing modes, org-html-export-to-html now exports an additional unicode character in source blocks. This thread has a workaround:

(defun org-html-fontify-code (code lang)
  ;; ...
  (funcall lang-mode)
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
  (when (require 'fill-column-indicator nil 'noerror)
    (fci-mode -1))
;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;;
  (insert code)
  ;; ...

Medium abuses nofollow

Nemo's Home — Mon, 06 Apr 2015 03:00:00 GMT

Update: Since I published this post, I have changed my opinion somewhat on the matter. This post is quite confrontational and I didn’t mean it to be that way. Medium is not wrong in this matter, but I still think we need to look for better solutions. I have since been working on a proposal/idea that would use machine learning to “solve” this problem, instead of side-stepping it.

I call medium a “mostly good” platform for lazy writers. A lot of people have written about its excellent typography, or it being the next “big publishing platform”. I’ve used medium in the past, and while it does have its benefits, I have stopped using it.

My primary reason was that I already have a blog, where I can control the entire experience. This is the same reason why New York Times does not start publishing articles on Medium.

The other reason is nofollow abuse.

Medium hosts more than 1M indexed pages. It has around 650k users currently by a conservative estimate. Rounding it to 700k to account for other users, collections, and other internal pages, it leaves us with around 300,000 articles on medium.

A basic tenet of the web is linkability. That is what Tim Berners Lee meant when he talked about HyperText:

HyperText is a way to link and access information of various kinds as a web of nodes in which the user can browse at will.

Over time, the web has evolved, and is now not just limited to human users, but to computers as well. This is an important consideration on which the web rests today. The biggest example of this is Google Search, which uses these links to “follow, spider, and index” the web. Google uses this linking information to build a citation index, which gives us the quality of a web page depending on the quality and number of sites that link to it.

If you know a bit or two about SEO, you might have heard of shady backlink techniques, which essentially amount to you getting links from an established site. This often takes the form of user-generated content such as comments and answers.

While fighting spam is important, it is far more important to make sure that web remains linked, that people are credited for the content they create. Medium hosts 300,000 articles published by half a million users, and yet none of these links back to external website, because of something called “rel=nofollow”.

When a link has a rel=nofollow attribute, search engines do not count it as a citation in their index. While this may be the right strategy for comments on a wordpress blog to prevent spam, this is not the right way to move forward if you want to “revolutionize the publishing industry”.

While medium is not as bad as some other sites in this regard (like quora, which even blocks the internet archive), it is very important because it portrays itself as a “publishing platform”. This means, medium is made up of articles, blog posts, with lots of outbound links compared to, for instance, StackOverflow answers (which solved this problem back in 2011).

If you publish content on medium, and provide relevant links for your readers, remember that these links are not considered as relevant by search engines.

Medium has said that this is not a top priority at the moment for them.

I understand completely. Handling spam would be a far more harder problem to solve than just blacklisting all outbound links. But we cannot go this way, if we want an open web. We need publishing platforms that cite content, and not blacklist it. This is why I write content on my own blog, and not on medium.

2015-04-05

ho.dges.online — Sun, 05 Apr 2015 03:00:00 GMT

Midnight on the Rest And Be Thankful. Probably the longest exposure I’ve done (10 mins!)

Lazy Performance Comparison of WordPress and Ghost

Anmol Sarma — Sat, 04 Apr 2015 21:40:03 GMT

So, I switched to a new blogging platform. Again. After trying out a static approach, I moved back to a WordPress when I stumbled across OpenShift (And their fantastic free plan). That was well over two years ago. In the meantime, I hardly blogged and didn’t really run into any issues but the blog did feel a tad sluggish.

Fast-forward to yesterday when I had a sudden impulse to try something new. Ghost looked interesting so, I spun up an install using OpenShift’s Ghost Quick-Start and proceeded to import data from my WordPress blog which turned out to be pretty painless.

Given the same resources (Small Gear) and same amount tweaking (i.e. none, using OpenShift’s quickstart installation defaults and using the default themes with no plugins), Ghost certainly felt a lot faster but how fast was it really? To answer that (And because I apparently have nothing better to do on a Friday night), I started a simultaneous stress test on WordPress and Ghost using BlazeMeter with a maximum of 50 users. The answer, as turns out is a helluva lot faster!

	Avg. Latency	Avg. Response Time	90%	95%	99%	Min	Max
Ghost	255.05	752.28	739	793	4647	602	7240
Wordpress	12129.55	21238.06	33641	37725	52178	2273	67093

On average, Ghost’s response is 28 times faster and its latency is 47 times lower. Looking at how the two respond to load is interesting:

Ghost doesn’t even seem to break a sweat while WordPress starts panting quite early into the run. The longest it took for Ghost to respond was 7240 ms which is almost a whole order of a magnitude faster than Wondpress’s 67093 ms. The shortest time for Ghost is 602 ms, a comfortable 3.77 times better than 2273 ms it took WordPress.

Basically Ghost crushes WordPress in this admittedly lazy and unscientific test. It is pretty similar to what others have reported. If your personal WordPress setup is struggling to keep up with traffic, you should at least consider Ghost. It doesn’t have anywhere near the number of plugins or themes as WordPress, it clearly is not as mature (It’s still in version 0.5.10) but boy is it fast! As for the writing flow, I can only say its different. I’m not entirely sure if it is better than what we have been used to (No spell check).

I for one am quite happy I tried (And switched to) Ghost. And I got a blog post out it to boot!

All hail Octocat!

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Sat, 04 Apr 2015 17:38:36 GMT

I am moving my personal projects to GitHub. While I had a good run with the simple gitweb interface, it just is not up-to-date any more. I hope to increase the reach of my projects via GitHub, in particular the QtOSG widget project.

You may find my main profile under http://github.com/Pseudomanifold. My previous project domain, git.annwfn.net has been set up to provide a permanent redirect to this profile.

Why Casting List(T) Isn't a Thing: Examining an 'obvious' upcast that really isn't.

Joshua Rogers — Sat, 04 Apr 2015 15:00:00 GMT

From time to time I hear a certain question floating around through the office, along the lines of “Why won't .NET let me pass my List<Thing> as a List<IThing>?” The tone of the question usually suggests that someone is annoyed that .NET isn't smart enough to make such a simple inference. The reality though is that the compiler is smart enough to know that it can't. You're kidding, right? If Thing is an IThing, then all the Thing in a List<Thing> must also be IThing.

Thinking through substrate

Liza Shulyayeva — Sat, 04 Apr 2015 13:21:27 GMT

After checking in the basic substrate item implementation yesterday I got to thinking - never a good sign.

The original idea was to have some pre-set types of substrate - garden dirt, rock, potting soil, pellets, whatever. But how would these items affect the snails and the jar? Would I need to invent some sort of set of attribute templates? Like “Oh well this garden dirt item is going to decrease jar temperature by 1C.” and “Oh well some snails somehow don’t like gardend dirt and prefer other substrates”. But why? Why would a snail prefer one template to another and how would this be decided?

2015-04-04

ho.dges.online — Sat, 04 Apr 2015 03:00:00 GMT

Breakfast for One.

The Poor Man's Character Controller

Evan Todd — Sat, 04 Apr 2015 00:00:00 GMT

Let's say that, like so many of us, you want to make a surreal voxel-based first-person parkour game. You're trying to figure out a production schedule. What will take the longest? Graphics? Sound? Level design? I bet it will be the character controller. And I bet it will take 4½ years. Why?

In running/jumping games, player movement is paramount. It takes forever to nail the right feeling.
Each game is a unique snowflake. You will not find an article explaining how to design the controls for your specific game. You're flying blind.

That said, each game offers a few transferrable bits of wisdom. Here's my story.

Try cocoapods out

xenodium.com @alvaro — Fri, 03 Apr 2015 03:00:00 GMT

Cocapods try:

$ pod try POD_NAME

Cornwall travel bookmarks

xenodium.com @alvaro — Fri, 03 Apr 2015 03:00:00 GMT

Food along the way.

Austria travel bookmarks

xenodium.com @alvaro — Fri, 03 Apr 2015 03:00:00 GMT

A 10 minutes walk into Grafana & Influxdb

Fabrice Aneche — Thu, 02 Apr 2015 19:49:04 GMT

This is a 10 minute tutorial to set up an InfluxDB + Grafana with Go on your Mac, but should work with minor modifcations on your favorite Unix too, it assumes you already have a working Go compiler. InfluxDB is a database specialized into time series, think store everything associated with a time, makes it perfect for monitoring and graphing values. Grafana is a js frontend capable of reading the data from InfluxDB and graphing it.

Java 8 and Functional Programming

GeekWare - Daniel Pecos Martínez — Thu, 02 Apr 2015 12:54:58 GMT

Purpose of this post is to provide a glimpse of the new features included in Java 8 that shift this language towards a more Functional Programming paradigm. But before, let’s define what we understand for Functional Programming (FP). Functional programming key characteristics include:

Higher Order Functions
Pure Functions and Immutability
Tail Call Recursion

Higher Order Functions for a FP language means that functions are considered first class citizens, allowing the programmer to use them as any other value the language defines, for example, a Function value:

can be assigned to a variable defined with a proper type
can be passed or returned to/from another functions, allowing functions to be composed or _curried_ (partially applied).

Pure Functions are those functions that don’t produce side effects. Instead of directly modifying data structures within the function body, those data structures are returned as result of the invocation. This behaviour results in Immutable data structures, which are far less error prone than classical mutable data structures.

Tail Call Recursion allows a compiler to reuse a call stack frame in a recursive algorithm, making the execution behave (in terms of memory allocation) as if it were an iterative algorithm. Instead of allocating new stack frames for every recursive call, if the structure of the call follows an specific pattern, the compiler then can reuse same frame, making possible the use recursion without any increase in program’s memory footprint.

So, what does Java 8 include in order to make FP paradigm closer to the programmer? The key feature included in this release is are Lambda Expressions.

But what’s a Lambda Expression? Well, it just a syntactic sugar included in the language, reducing the amount of code needed to work with anonymous classes. Lambdas’ its syntax kind of reminds of a closure or monad, but let’s better see an example of a lambda:

 System.out.println("Hello Lambdas!");

But, as previously said, this could also be defined as anonymous class, as this is just syntactic sugar (let’s assume that LambadInterface is an already defined interface with one method apply):

new LambdaInterface {
   @Override
   public void apply() {
      System.out.println("Hello... Lambdas?");
   }
};

Not quite the same? Let’s evolve these pieces of code a little bit further:

 System.out.println("Hello Lambdas!");
lamdba.apply(); // Hello Lambdas!

LambdaInterface lambda = new LambdaInterface {
   public void apply() {
      System.out.println("Hello... Lambdas?");
   }
};
lamdba.apply(); // Hello... Lambdas?

How do Lambdas differ from FP Closures? Well, a closure is an anonymous function packed with the context where it was defined, being able to modify it’s context (context defined variables are, in fact, closed) . Java 8 Lambdas can access external context variables but only if they’re defined as final or if the programmer assures that they are not going to change their value (are actually final).

We said that Function Composition is also another key characteristic of FP, so, are Lambdas composable? Let’s see an actual example of Lambdas composition in action:

 n * 2;
Function&lt;Integer, Integer&gt; increment = (n) -> n + 1;

Function&lt;Integer, Integer&gt; doublePlusOne = duplicate.andThen(increment);

Unfortunately, Tail recursion is not included in Java 8, so we’ll still have to be cautious with memory consumption in recursive algorithms.

In future posts, we’ll deep into Lambdas and Functional Interfaces, and how they allow us to create a cleaner and more expressive code.

Jar substrate is a go

Liza Shulyayeva — Thu, 02 Apr 2015 11:41:21 GMT

It’s finally in. Well, the beginning stages anyway. Jar substrate. Anyone who knows anything about snails knows that they can’t live in a glass box without some sort of substrate to hide in, get moisture from, and burrow under. So now you can apply one type of substrate (so far) to a jar. Eventually there will be different kinds of substrate, and maybe even substrate you can mix together from other types of substrate. Currently the substrate has no effect on the snails or the environment outside of being present and visually represented, but eventually it can impact everything from snail nutrients to jar temperature and what kinds of plants you can grow inside the jar. Here’s how it works.

Tackling Gastropoda's memory usage - Round One

Liza Shulyayeva — Thu, 02 Apr 2015 10:20:27 GMT

I’ve been having to power cycle my Gastropoda Digital Ocean droplet every day lately because something was hogging up all the memory. It was a little annoying to diagnose because I know the problem had to do with the recurring events that are run using a cron job, of which there are quite a few. I know ideally PHP isn’t something you would use for a long running server process, but these jobs weren’t long running - they were just frequent.

Cinnamon desktop run dialog

xenodium.com @alvaro — Thu, 02 Apr 2015 03:00:00 GMT

Note to self. Open with Alt-f2.

Antimony

Matt Keeter — Thu, 02 Apr 2015 03:00:00 GMT

A fresh look at CAD software

Protecting Users from Phishing and Fraud

Brightball Articles — Thu, 02 Apr 2015 01:47:23 GMT

This presentation covers my experiences combatting phishing and fraud using DMARC and assorted other techniques in a large eBay-like platform for a niche market...when the site previously did everything over direct user email...for over a decade. Good times.

Talk Python To Me Podcast Episode #1: EVE RESTful APIs for Humans

Nicola Iarocci — Wed, 01 Apr 2015 03:00:00 GMT

I was lucky enough be the first guest for the shiny new Talk Python To Me Podcast hosted by Michael Kennedy. In this episode we talk about Eve an my other open source releases, which gives us an excuse to touch on a variety of topics such as Polyglot Programming, New Microsoft and the .NET evolution, MongoDB and the Open Source eco-system as seen from the point of view of an old fart who has been spending most of his career in closed systems.

Google Drive: Publicly hosting a static website

Wed, 01 Apr 2015 03:00:00 GMT

I frequently need to demo work in progress web sites and web apps to people I’m working with. I was looking for the most convenient way to do this and found that Google Drive was a good fit to letting me publicly share web development projects. Google Drive provides an easy way to store and share documents in the cloud. The Google Drive desktop app also conveniently keeps a folder on your machine in sync with your online Google Drive folder in the same way Dropbox does.

DRY Configuration

Craig Pardey — Mon, 30 Mar 2015 03:00:00 GMT

Externalizing application configuration values into a properties file is a common pattern in enterprise applications, because it gets the environment-specific settings out of the code and avoids the need for environment-specific builds. Being able to deploy the same binary to multiple environments (Dev, QA, UAT, Production) is a Good Thing.

I came across an interesting quirk in an application recently. I needed to change the port of the application server for one of the app’s components, and the port value was wired up via Spring using some code that looked like this:

Sqlplus is my second home, part 8: Embedding multiple sqlplus arguments into one variable

Tanel Poder Blog — Mon, 30 Mar 2015 00:23:51 GMT

I’ve updated some of my ASH scripts to use these 4 arguments in a standard way:

What ASH columns to display (and aggregate by)
Which ASH rows to use for the report (filter)
Time range start
Time range end

So this means whenever I run ashtop (or dashtop) for example, I need to type in all 4 parameters. The example below would show top SQL_IDs only for user SOE sessions from last hour of ASH samples:

Books for 2015

xenodium.com @alvaro — Sun, 29 Mar 2015 02:00:00 GMT

Ayahuasca bookmarks

xenodium.com @alvaro — Sun, 29 Mar 2015 02:00:00 GMT

On proper git commit messages and frequent git pushes

Erethon's Corner — Sat, 28 Mar 2015 15:37:51 GMT

Edit: This post was written many years ago. Back then I was a lot more strict on how other people did things and I now understand this was a mistake. I'm leaving this post here as a reminder of past mistakes.

Note: this is a rant on how people use git.

When I was introduced to git, I was told I should read this great post on git commit messages by Tim Pope. Ever since, I've been trying to follow that system in all of my commits across all projects.

Emacs init.el bookmarks

xenodium.com @alvaro — Sat, 28 Mar 2015 02:00:00 GMT

CSS vertical align using flex

xenodium.com @alvaro — Sat, 28 Mar 2015 02:00:00 GMT

Codepen snippet:

div{
  height: 200px;
  background: #ccc;
  display: flex;
  justify-content: center;
  align-items: center;
}

p{
  margin: auto
}

[3.27.15] Dell 3007 Repair

transistor-man.com — Fri, 27 Mar 2015 10:38:38 GMT

A quick 'upgrade' of a dell 3007 monitor to address power supply brownout due to thermal loading.

Team Culture by Merlin Mann

ho.dges.online — Fri, 27 Mar 2015 02:00:00 GMT

Most team culture comes out of a combination of what is tolerated and what is rewarded. If you legit want your culture to improve, change what you reward and rethink what you will tolerate.

How Celery Chord Synchronization Works

Untrod — Thu, 26 Mar 2015 19:20:00 GMT

Celery is a powerful tool for managing asynchronous tasks in Python. The basic model is synchronous Python code pushes a task (in the form of a serialized message) into a message queue (the Celery "broker", which can be a variety of technologies - Redis, RabbitMQ, Memcached, or even a database), and …

London diving schools

xenodium.com @alvaro — Thu, 26 Mar 2015 02:00:00 GMT

London Diving School. Not heard good comments.
Dive Wimbledon. Heard ok comments.
Clidive is a BSAC club (amateur organisation). Not commercial but may take longer.
Sublime Diving. Heard good comments.
Oyster Diving. Pool in cetral London. Good comments.
SS Thistlegorm mentioned as a memorable site.
Many suggest to get certified elsewhere. Perhaps Egypt via Poseidon Divers.

Helm buffer URLs

xenodium.com @alvaro — Thu, 26 Mar 2015 02:00:00 GMT

Venturing into Emacs lisp and Helm. Here's a go at listing all URLs in current buffer.

(require 'goto-addr)

(defun ar/helm-buffer-url-candidates ()
  "Generate helm candidates for all URLs in buffer."
  (save-excursion
    (goto-char (point-min))
    (let ((helm-candidates '())
          (url))
      (while (re-search-forward goto-address-url-regexp
                                nil t)
        (setq url
              (buffer-substring-no-properties (match-beginning 0)
                                              (match-end 0)))
        (add-to-list 'helm-candidates
                     (cons url
                           url)))
      helm-candidates)))

(defun ar/helm-buffer-urls ()
  "Narrow down and open a URL in buffer."
  (interactive)
  (helm :sources `(((name . "Buffer URLs")
                    (candidates . ,(ar/helm-buffer-url-candidates))
                    (action . (lambda (url)
                                (browse-url url)))))))

Groovy to be part of the Apache Software Foundation

GeekWare - Daniel Pecos Martínez — Wed, 25 Mar 2015 20:06:15 GMT

As March 11th, Pivotal dropped its financial sponsorship for Groovy, and despite of not really endangering it, as Groovy is an already well established language with a great community backing it, it raised many concerns, as the required boost a platform like this deserves was missing until now. Furthermore, its creator and project leader until lately, Guillaume Laforge, also recently stepped back in order to focus in Restlet.

Also the recent release of Java 8, with the introduction of lambdas into the Java language has increased the interest and traction of Java, making a lot of people question if there were still room for other JVM languages, even more being so closed to Java.

So being worried about Groovy’s future is a normal consequence of this recent events. But don’t forget that this language has been community driven for a long time now, making this changes have a lower impact than initially expected.

So in order to clarify them, moving to the Apache Software Foundation has been a really smart move by the community, and as Cédric Champeu explains, a natural next step in Groovy’s lifetime that will help it weathering those changes.

This backup guarantees the position Groovy has as a trending alternative inside the JVM, making it still the most successful productivity booster and most enjoyable language, compared to the Old Java Language, within the JVM.

Further references:

http://groovy.dzone.com/articles/groovy-submitted-apache

Things That Have Almost Killed Me

JonPaulUritis.com — Wed, 25 Mar 2015 02:00:00 GMT

I’m celebrating my 30th birthday November 4th. I am completely flabbergasted that I have lived this long. So in honor of my 30th I’d like to put out to the world a list of things that could have killed me, or could have led to situations that could have killed me.

Fell off a cliff at a rock quarry into a foot and a half of water
Broke my back snowboarding (later broke my clavicle snowboarding)
Fist fight at our Christmas formal that Heavy B saved me from having a 1.75 bottle of Jack Daniels smashed through the back of my head.
Having a old guy crash through the window into our retail location with his car.
3 car accidents
1 near car accident because we were having a 6 person Nerf gun fight in a mini-van I was driving, but its okay because we won
Almost hit by a NYC subway
Food poisoning in Tianjin China.
Almost drowned surfing hurricane swell at Salt Creek (double overhead)
Guy at Johnny Brenda’s in Fishtown pulled his gun on us
Almost drowned by pod of dauphins
Hit a deer with a car in Minnesota
United almost crashing the plane on my way home to California (totally not kidding) last December
(New! 2023) Guy at a basketball game threatened me and my team mates with a gun, SWAT team was called in full blazing AR-15s and Shotguns. The guy slipped out the back door.
Not really life threatening but I feel like its worth mentioning I was almost Deported back to Canada (I have dual-citizenship)
Fell off a rooftop in Philadelphia

Oracle Exadata Performance: Latest Improvements and Less Known Features

Tanel Poder Blog — Tue, 24 Mar 2015 16:57:51 GMT

Here are the slides of a presentation I did at the IOUG Virtual Exadata conference in February. I’m explaining the basics of some new Oracle 12c things related to Exadata, plus current latest cellsrv improvements like Columnar Flash Cache and IO skipping for Min/Max retrieval using Storage Indexes:

Note that Christian Antognini and Roger MacNicol have written separate articles about some new features:

Enjoy!

VoCore OpenWRT as NAT access point

Allan's Blog — Tue, 24 Mar 2015 02:43:53 GMT

I recently bought a VoCore. It came pre-flashed with OpenWRT Chaos Calmer (as of 2015-03-16). OpenWRT was configured as a wireless access point, bridging with the ethernet port. Here is what I did to get it to a configuration where it acts as a NAT'ing wireless access point, similar to most consumer routers/access points:

SSH'ed to its default IP of 192.168.61.1.

Edited /etc/config/network:

...
#config interface 'lan'
config 'interface' 'wan'
      option macaddr 'b8:d8:12:60:00:01'
      option proto 'dhcp'

config interface 'lan'
  option force_link '1'
  option macaddr 'b8:d8:12:60:00:02'
  option proto 'static'
  option ipaddr '192.168.61.1'
  option netmask '255.255.255.0'
...

Edited /etc/config/wireless:

...
config wifi-iface
      option device   radio0
      option network  lan
      option mode     ap
      option ssid     gaffel8080
      option encryption psk2
      option key palle123
...

Edited /etc/config/dhcp:

...
#config odhcpd 'odhcpd'
# option maindhcp '0'
# option leasefile '/tmp/hosts/odhcpd'
# option leasetrigger '/usr/sbin/odhcpd-update'
...

To enable SSH from the ethernet port, I also edited /etc/config/firewall:

...
config rule
      option src wan
      option proto tcp
      option dst_port 22
      option target ACCEPT

This was all that was needed for the basic scenario of using VoCore as an accesspoint combined with a NAT'ing router.

I went a little further and installed some nice to have packages like openssh-sftp-server, nano, htop, ip, etc. However, in order to do this, I first had to fix /etc/opkg.conf:

...
#src/gz cheese_base http://downloads.openwrt.org/snapshots/trunk/ramips/packages/base
#src/gz cheese_luci http://downloads.openwrt.org/snapshots/trunk/ramips/packages/luci
src/gz chaos_calmer_base http://downloads.openwrt.org/snapshots/trunk/ramips/generic/packages/base/
src/gz chaos_calmer_luci http://downloads.openwrt.org/snapshots/trunk/ramips/generic/packages/luci/
src/gz chaos_calmer_management http://downloads.openwrt.org/snapshots/trunk/ramips/generic/packages/management/
src/gz chaos_calmer_packages http://downloads.openwrt.org/snapshots/trunk/ramips/generic/packages/packages/
src/gz chaos_calmer_routing http://downloads.openwrt.org/snapshots/trunk/ramips/generic/packages/routing/
src/gz chaos_calmer_telephony http://downloads.openwrt.org/snapshots/trunk/ramips/generic/packages/telephony/

After that I could install the extra packages by running the following:

opkg update
opkg install openssh-sftp-server
opkg install nano
opkg install htop
opkg install ip

Also, I recommend disabling Luci, as it is buggy, and a security concern:

rm /www/cgi-bin/luci
echo "A private box" > /www/index.html

Darknet Market Basket Analysis

Ryan Compton — Tue, 24 Mar 2015 02:00:00 GMT

The Evolution darknet marketplace was an online black market which operated from January 2014 until Wednesday of last week when it suddenly disappeared. A few days later, in a reddit post, gwern released a torrent containing daily wget crawls of the site dating back to its inception. I ran some off-the-shelf affinity analysis on the dataset – here’s what I found: Products can be categorized based on who sells them On Evolution there are a few top-level categories (“Drugs”, “Digital Goods”, “Fraud Related” etc.) which are subdivided into product-specific pages. Each page contains several listings by various vendors. I built a graph between products based on vendor co-occurrence relationships, i.e. each node corresponds to a product with edge weights defined by the number of vendors who sell both incident products. So, for example, if there are 3 vendors selling both mescaline and 4-AcO-DMT then my graph has an edge with weight 3 between the mescaline and 4-AcO-DMT nodes. I used graph-tool’s implementation of stochastic block model-based hierarchal edge bundling to generate the below visualization of the Evolution product network: The graph is available in graphml format here. It contains 73 nodes and 2,219 edges (I found a total of 3,785 vendors in the data). Edges with higher weights are drawn more brightly. Nodes are clustered with a stochastic block model and nodes within the same cluster are assigned the same color. There is a clear division between the clusters on the top half of the graph (correpsonding to drugs) and the clusters on the bottom half (corresponding to non-drugs, i.e. weapons/hacking/credit cards/etc.). This suggests that vendors who sold drugs were not as likely to sell non-drugs and vice versa.

Java 8 one year later

GeekWare - Daniel Pecos Martínez — Mon, 23 Mar 2015 20:48:20 GMT

It’s been a little more than a year since Java 8 was released (2014/03/18) and you might think that it’s a little too late for a What’s new in post. In fact latest public update available is 8u40, so let’s review not only what was initially included in Java 8, but what else has changed during this first year, up to release 8u40.

Lots of changes were included in the initial Java 8 release, being probably the most notable of them, in my opinion (feel free to disagree, looking forward to discussions):

Java Language
- Lambda Expressions
- Method references
- Default methods
- Functional interfaces
- Method parameter reflection
- Optional references
API
- Collections Streams API
- Date-Time API (based on the popular Joda time library)
- Parallel Array Sorting
- Base64 encoding and decoding
- Unsigned Arithmetic Support
Tools
- Nashorn Script Engine – Javascript in the JVM
- Java Mission Control

… among many other optimizations, fixes and new capabilities. You can get a fully detailed changelog detail here.

We’re not going to go into further detail for these new characteristics, as you’ll find plenty of articles about them. Furthermore some of those would require entire books to proper explain and deal with them, and that’s not the purpose of this article.

But let’s review what has changed during this first year of Java 8’s life (up to 8u40 release). Again, these are the most important in my opinion:

New jdeps analysis tool
Advanced Management Console (AMC)
New Garbage Collection Tuning Guide added to JDK 8 documentation
SSLv3 disabled by default
Java Flight Recorder can be enabled in runtime

… and many other improvements and bugfixes.

And last but no least, we should talk about Java 8 adoption rate, which has been surprisingly higher than initially expected based on a Typesafe’s survey:

Interestingly, one of the conclusions extracted from that survey is that the inclusion of Lambdas in the Java language has not hurt Scala at all, but validated its approach to Functional Programming:

Other references:

Dell System Detect RCE vulnerability

Tom Forbes — Mon, 23 Mar 2015 17:16:10 GMT

I recently discovered a serious flaw with Dell System Detect that allowed an attacker to trigger the program to download and execute an arbitrary file without any user interaction. Below is a summary of the issue and the steps taken to bypass the protections Dell put in place. Timeline: The issue wa...

Chromecasts, Netflix & UI-200

Joseph Price — Mon, 23 Mar 2015 02:00:00 GMT

My Chromecast has regularly been refusing to play Netflix streams recently with error ui-200.

Doh! undo last git commit

xenodium.com @alvaro — Mon, 23 Mar 2015 02:00:00 GMT

$ git reset --soft HEAD~1

BSDF System

Code & Visuals — Mon, 23 Mar 2015 02:00:00 GMT

Takua a0.5’s BSDF system was particularly interesting to build, especially because in previous versions of Takua Renderer, I never really had a good BSDF system. Previously, my BSDFs were written in a pretty ad-hoc way and were somewhat hardcoded into the pathtracing integrator, which made BSDF extensibility very difficult and multi-integrator support nearly impossible without significant duplication of BSDF code. In Takua a0.5, I’ve written a new, extensible, modularized BSDF system that is inspired by Mitsuba and Renderman 19/RIS. In this post, I’ll write about how Takua a0.5’s BSDF system works and show some pretty test images generated during development with some interesting models and props.

First, here’s a still-life sort of render showcasing a number of models with a number of interesting materials, all using Takua a0.5’s BSDF system and rendered using my VCM integrator. All of the renders in this post are rendered either using my BDPT integrator or my VCM integrator.

BSDFs in Takua a0.5 are designed to support bidirectional evaluation and importance sampling natively. Basically, this means that all BSDFs need to implement five basic functions. These five basic functions are:

Evaluate, which takes input and output directions of light and a normal, and returns the BSDF weight, cosine of the angle of the input direction, and color absorption of the scattering event. Evaluate can also optionally return the probability of the output direction given the input direction, with respect to solid angle.
CalculatePDFW, which takes the input and output directions of light and a normal, and returns the forward probability of the output direction given the input direction. In order to make the BSDF operate bidirectionally, this function also needs to be able to return the backwards probability if the input and output are reversed.
Sample, which takes in an input direction, a normal, and a random number generator and returns an output direction, the BSDF weight, the forward probability of the output direction, and the cosine of the input angle.
IsDelta, which returns true if the BSDF’s probability distribution function is a Dirac delta function and false otherwise. This attribute is important for allowing BDPT and VCM to handle perfectly specular BSDFs correctly, since perfectly specular BSDFs are something of a special case.
GetContinuationProbability, which takes in an input direction and normal and returns the probability of ending a ray path at this BSDF. This function is used for Russian Roulette early path termination.

In order to be correct and bididirectional, each of these functions should return results that agree with the other functions. For example, taking the output direction generated by Sample and calling Evaluate with the Sample output direction should produce the same color absorption and forward probability and other attributes as Sample. Sample, Evaluate, and CalculatePDFW are all very similar functions and often can share a large amount of common code, but each one is tailored to a slightly different purpose. For example, Sample is useful for figuring out a new random ray direction along a ray path, while Evaluate is used for calculating BSDF weights while importance sampling light sources.

Small note: I wrote that these five functions all take in a normal, which is technically all they need in terms of differential geometry. However, in practice, passing in a surface point and UV and other differential geometry information is very useful since that allows for various properties to be driven by 2D and 3D textures. In Takua a0.5, I pass in a normal, surface point, UV coordinate, and a geom and primitive ID for future PTEX support, and allow every BSDF attribute to be driven by a texture.

One of the test props I made is the PBRT book, since I thought rendering the Physically Based Rendering book with a physically based renderer and physically based shading would be amusing. The base diffuse color is driven by a texture map, and the interesting rippled and variation in the glossiness of the book cover comes from driving additional gloss and specular properties with more texture maps.

In order to be physically correct, BSDFs should also fulfill the following three properties:

Positivity, meaning that the return value of the BSDF should always be positive or equal to 0.
Helmholtz Reciprocity, which means the return value of the BSDF should not be changed by switching the input and output directions (although switching the input and output CAN change how things are calculated internally, such as in perfectly specular refractive materials).
Energy Conservation, meaning the surface cannot reflect more light than arrives.

At the moment, my base BSDFs are not actually the best physically based BSDFs in the world… I just have Lambertian diffuse, normalized Blinn-Phong, and Fresnel-based perfectly specular reflection/refraction. At a later point I’m planning on adding Beckmann and Disney’s Principled BSDF, and possibly others such as GGX and Ward. However, for the time being, I can still create highly complex and interesting materials because of the modular nature of Takua a0.5’s BSDF system; one of the most powerful uses of this modular system is combining base BSDFs into more complex BSDFs. For example, I have another BSDF called FresnelPhong, which internally calls normalized Blinn-Phong BSDF but also calls the Fresnel code from my Fresnel specular BSDF to account for an output direction with the Fresnel effect with glossy surfaces. Since the Fresnel specular BSDF handles refractive materials, FresnelPhong allows for creating glossy transmissive surfaces such as frosted glass (albeit not as accurate to reality as one would get with Beckmann or GGX).

Another one of my test props is a glass chessboard, where half of the pieces and board squares are using frosted glass. Needless to say, this scene is very difficult to render using unidirectional pathtracing. I only have one model of each chess piece type, and all of the pieces on the board are instances with varying materials per instance.

Another interesting use of modular BSDFs and embedding BSDFs inside of other BSDFs is in implementing bump mapping. Takua a0.5 implements bump mapping as a simple BSDF wrapper that calculates the bump mapped normal and passes that normal into whatever the underlying BSDF is. This approach allows for any BSDF to have a bump map, and even allows for applying multiple bump maps to the same piece of geometry. In addition to specifying bump maps as wrapper BSDFs, Takua a0.5 also allows attaching bump maps to individual geometry so that the same BSDF can be reused with a number of different bump maps attached to a number of different geometries, but under the hood this system works exactly the same as the BSDF wrapper bump map.

This notebook prop’s leathery surface detail comes entirely from a BSDF wrapper bump map:

Finally, one of the most useful and interesting features of Takua a0.5’s BSDF system is the layered BSDF. The layered BSDF is a special BSDF that allows arbitrary combining, layering, and mixing between different BSDFs, much like Vray’s BlendMtl or Renderman 19/RIS’s LM shader system. Any BSDF can be used as a layer in a layered BSDF, including entire other layered BSDF networks. The Takua layered BSDF consists of a base substrate BSDF, and an arbitrary number of coat layers on top of the substrate. Each coat is given a texture-drive weight which determines how much of the final output BSDF is from the current coat layer versus from all of the layers and substrate below the current coat layer. Since the weight for each coat layer must be between 0 and 1, the result layered BSDF maintains physical correctness as long as all of the component BSDFs are also physically correct. Practically, the layered BSDF is implemented so that with each iteration, only one of the component BSDFs is evaluated and sampled, with the particular component BSDF per iteration chosen randomly based on each component BSDF’s weighting.

The layered BSDF system is what allows the creation of truly interesting and complex materials, since objects in reality often have complex materials consisting of a number of different scattering event types. For example, a real object may have a diffuse base with a glossy clear coat, but there may also be dust and fingerprints on top of the clear coat contributing to the final appearance. The globe model seen in my adaptive sampling post uses a complex layered BSDF; the base BSDF is ground glass, with the continents layered on top as a perfectly specular mirror BSDF, and then an additional dirt and fingerprints layer on top made up of diffuse and varying glossy BSDFs:

Here’s an additional close-up render of the globe that better shows off some of the complex surface detail:

Going forward, I’m planning on adding a number of better BSDFs to Takua a0.5 (as mentioned before). Since the BSDF system is so modular and extensible, adding new BSDFs should be relatively simple and should require little to no additional work to integrate into the renderer. Because of how I designed BSDF wrappers, any new BSDF I add will automatically work with the bump map BSDF wrapper and the layered BSDF system. I’m also planning on adding interesting effects to the refractive/transmission BSDF, such as absorption based on Beer’s law and spectral diffraction.

After I finish work on my thesis, I also intend on adding more complex materials for subsurface scattering and volume rendering. These additions will be much more involved than just adding GGX or Beckmann, but I have a rough roadmap for how to proceed and I’ve already built a lot of supporting infrastructure into Takua a0.5. The plan for now is to implement a unified SSS/volume system based on the Unified Points, Beams, and Paths presented at SIGGRAPH 2014. UPBP can be thought of as extending VCM to combine a number of different volumetric rendering techniques. I can’t wait to get started on that over the summer!

Achieving Full Marks on Qualys SSL Labs

Dan Palmer — Mon, 23 Mar 2015 02:00:00 GMT

Qualys have become well known in the recent crop of SSL and TLS vulnerabilities as a first-responder with automated testing and validation, but scoring top marks on their SSL Labs test can be difficult. I explored what was required to score full marks.

Resetting variables using defvar

xenodium.com @alvaro — Sun, 22 Mar 2015 02:00:00 GMT

Want to re-evaluate defvars and modify variables? eval-defun (bound to C-M-x) can help. From the manual:

If the current defun is actually a call to `defvar', then reset the variable using its initial value expression even if the variable already has some other value. (Normally `defvar' does not change the variable's value if it already has a value.) Treat `defcustom' similarly.

Broken Xcode plugins?

xenodium.com @alvaro — Sun, 22 Mar 2015 02:00:00 GMT

Some Xcode plugins stopped loading after updating Xcode. Ensure the latest DVTPlugInCompatibilityUUIDs is added to the plugin's Info.plist. Get from:

$ defaults read \
    /Applications/Xcode.app/Contents/Info DVTPlugInCompatibilityUUID

Additional suggestions as Stack Overflow.

Born to Run references

xenodium.com @alvaro — Sun, 22 Mar 2015 02:00:00 GMT

References from reading Christopher McDougall's Born to Run:

Chi running.
Chia and pinole. Random recipes here.
Heritage Seeds.
Ken Mierke’s Evolution running.
Pose method.
Quotes, references, videos, etc. at ultimate paleo guide.

Recipe by Megan Mignot, based on book references:

Mama Tita’s Pancakes

1 ½ cups cooked brown rice
1 cup coconut milk
2 ripe bananas
1 tablespoon honey
½ cup white cornmeal
2 teaspoons baking powder

Dallas Buyers Club

🦉 olshansky 🦁 — Sat, 21 Mar 2015 08:03:18 GMT

Olshansky's review of Dallas Buyers Club

Tinker Tailor Soldier Spy

🦉 olshansky 🦁 — Sat, 21 Mar 2015 07:52:31 GMT

Olshansky's review of Tinker Tailor Soldier Spy

Base Blitz

mbutler — Sat, 21 Mar 2015 03:57:35 GMT

The 4th edition of Dungeons & Dragons often gets a bad rap for being too video gamey with encounters that drag out forever. However, it’s a very elegant and well-designed game that balances combat well, especially playing on a grid. It’s been my feeling that 4e would make an excellent video game. More specifically, 4e […]

read, _read, and readable: Readable Streams in Node.js

Khanna Law Blog — Sat, 21 Mar 2015 02:00:00 GMT

This article explains how readable streams work by considering the relationship between `read()`, `_read()`, and `push()` methods, and the `readable` event defined on the Readable base class

Monk is now a Ruby app.

Ben Overmyer's Site — Sat, 21 Mar 2015 02:00:00 GMT

Well, it happened. It took a few days of work, but I taught myself enough Ruby on Rails to fully refactor Monkblog from PHP/Laravel to RoR.

The blog is now running on Heroku, and my next step will be to setup continuous deployment. I'm pretty excited about this development! I'm officially running a Ruby on Rails app in production that I wrote myself.

And of course, now that that's done, I'm working on improvements to the blog. The first order of business will probably be to make the writing screen look and feel more like Ghost's marvelous editor.

I also need to port all of my old content from wherever I can find it into this new blog. I lost several posts when my old web server died (long story), but I imagine I can restore them easily enough. Just need to find backups...

2015-03-21

ho.dges.online — Sat, 21 Mar 2015 02:00:00 GMT

Repairing a Microsoft ergonomic keyboard

zufallsheld — Sat, 21 Mar 2015 01:00:00 GMT

I accidentally spilled tea on my Microsoft natural ergonomic keyboard (Referal link). After cleaning up the visible mess I continued to use the keyboard as normal as everything still worked. But the next day the keyboard didn’t work anymore. The lights were still flashing but the keys I pressed …

Welcome to jvmGeek!

GeekWare - Daniel Pecos Martínez — Fri, 20 Mar 2015 15:09:42 GMT

Welcome to jvmGeek!

This new blog aims to talk and discuss about the JVM ecosystem, with news and articles discussing about Java – the language -, and also about other JVM languages such as Scala, Kotlin or Clojure.

But it won’t deal only about programming languages, but also about tools and libraries. You’ll find news, tutorials, code examples and more general articles, among other.

From time to time I will also try to gather events and news related to one particular matter and post them as a flash post that will help you keep up to date with this effervescent platform.

I hope you’ll enjoy reading this blog as least as much as I did writing it!

Thank you very much!

Daniel Pecos (@danielpecos).

Blog post on recent talk

Nemo's Home — Fri, 20 Mar 2015 02:00:00 GMT

So I recently did a talk on Joy of Software Development. You can read more about the talk here (link includes slides and list of topics covered). This post is devoted to the references I’d promised to link to in the talk. Since it was an introductory talk, and I didn’t want to bore people to death, I decided to cover lots of topics at a shallow depth, instead of covering a few topics deeply.

This means that I need to post more material for people to follow up on. So, this is that reference blog post. Make sure you have a copy of the slides open as you go through these links.

Update: I also gave this talk (with a few updates) at GeekSkool in October 2015.

Software Development in general

Hiring in software industry is broken
Breadth First Learning
Recommended readings for developers by Jeff Atwood (codinghorror)
Ship v1, even if it sucks
Ship early, ship often
Never rewrite, always refactor by Joel Spolsky
The Joel Test to score a software company.

Software Security

Security is the opposite of obscurity
OWASP Top 10
Software Security course on coursera
CTFTime - See upcoming CTF contests
SmashTheStack - Learn buffer overflow attacks
Nebula Exploit Excercises
Backdoor - Security CTF platform for beginners by SDSLabs

Starting Advice

Agnostic Software Development

Free and open source development

Why Open Source misses the point of Free Software - by Richard Stallman
Open source definition
Choose a license
What is free software?
Why Free Software Movement is important
Mozilla mission statement

Version Control

Tests

During the talk, I decidedly used the term TDD incorrectly. TDD technically means going test first, but I used it as an introduction to testing in general. This was intentional. The links here will use TDD in the correct sense.

REST and APIs

Unix Philosophy

The epic Knuth vs McIlroy story
Zawinski’s Law
http://onethingwell.org/
Wikipedia article on the topic is surprisingly good
Good examples of pipes

Books

These are books i absolutely recommend every software developer to read, in order.

Other than these, I recommend reading Code Complete, Mythical Man Month, and everything by Jeff Atwood and Zach Homan, but only after you have read the above 2 books.

How to get better at Software Development?

This is just a small list of topics I cover in a recent blog post. This is only present in the updated version of the talk which I gave at GeekSkool. You can read the blog post here to look at the points I make.

Phew. That was a lot of links. If you are ever interested in learning more about software development, feel free to contact me. If you ever feel like chatting with me, I’m usually online at chat.sdslabs.co.

Org tips from 2015-03-18 Emacs hangout

xenodium.com @alvaro — Thu, 19 Mar 2015 02:00:00 GMT

Lots of great tips in Emacs Hangout 2015-03-18. Favorites:

Private org drawer to prevent export:

:PRIVATE:
My super duper secret text I don't want to export.
:END:

C-c C-p/C-c C-n Jump over sections.
(setq org-hide-leading-stars t).
(org-bullets-mode).
(org-refile).
C-u (org-refile) jumps, no refile.
(helm-org-in-buffer-headings).

Food bookmarks

xenodium.com @alvaro — Thu, 19 Mar 2015 02:00:00 GMT

Ethiopia travel bookmarks

xenodium.com @alvaro — Thu, 19 Mar 2015 02:00:00 GMT

Bet Giyorgis Church.

China travel bookmarks

xenodium.com @alvaro — Thu, 19 Mar 2015 02:00:00 GMT

"If you have not been to Kashgar, then you haven’t really been to Xinjiang”.
7 first-timer fails to avoid on your trip to Beijing.
Archpng on Twitter: "Mount Maiji Grottoes,China… ".
Beijing on a budget.
Chinese cities you've never heard of.
Ctrip for hotels. Stick to high-rated only.
Essential regional cuisine of China.
Fenghuang ancient town.
Hallelujah Mountains.
Houhai Lake (Beijing). Miscellaneous bars.
Jiuzhaigou nature reserve.
Luotuofeng peak, Sichuan.
Mount Huashan.
Mount Maiji Grottoes (DuckDuckGo).
Nan Luo Gu Xiang (Beijing). Street Food!
Qianmen Street. Near Tiananmen Square. Artifact shopping and famous food.
Rock pools.
Shanghai Street Food #7 Jiān Bǐng 煎餅.
Shenzhen (Hong Kong). The worlds manufacturing ecosystem.
Suspended Temple of Mt. Hengshan.
The essential guide to backpacking China's silk road.
What are some must-try foods when visiting China? (Quora).
Zhangjiajie National Forest Park.
China's most epic high-speed rail journeys.

Updates to searchcode.com

Ben E. C. Boyter — Thu, 19 Mar 2015 00:11:33 GMT

Just a quick post to list some updates to searchcode.com The first is a slight modification to the home page. A while ago I received an email from the excellent Christian Moore who provided some mock-ups of how he felt it should look. I loved the designs, but was busy working on other issues. Thankfully however in the last week or so I found the time to implement his ideas and the result is far more professional to me.

The month from Hell

Liza Shulyayeva — Wed, 18 Mar 2015 17:59:25 GMT

Update on Rigel and response from DjurAkuten

Update: When I first wrote this post I kept all clinic names anonymous. I am now un-anonymising one of the clinics (DjurAkuten), where this whole chain of events was triggered in the first place and which is now trying to claim that Rigel was already sick/had a flaccid bladder when we brought him in for a routine surgery as a healthy kitten despite their own records stating otherwise.

South Korea travel bookmarks

xenodium.com @alvaro — Wed, 18 Mar 2015 02:00:00 GMT

Sharing on iOS

xenodium.com @alvaro — Wed, 18 Mar 2015 02:00:00 GMT

UIActivityViewController.
- Use completionWithItemsHandler on iOS 8.
- Sample:

NSString *title = @"Sharing on iOS bookmarks.";
NSURL *url = [NSURL URLWithString:@"http://xenodium.com/#sharing-on-ios"];
UIImage *image = [UIImage imageNamed:@"beautiful-image"];

UIActivityViewController *controller =
  [[UIActivityViewController alloc]
    initWithActivityItems:@[title, url, image]
    applicationActivities:nil];

// self being a UIViewController.
[self presentViewController:controller animated:YES completion:nil];

Sharing through Mail app on simulator isn't supported.

viewServiceDidTerminateWithError: Error Domain=_UIViewServiceInterfaceErrorDomain Code=3 "The operation couldn’t be completed. (_UIViewServiceInterfaceErrorDomain error 3.)" UserInfo=… {Message=Service Connection Interrupted}

Sharing through Mail app on device is OK.
WWDC 2014 Creating Extensions for iOS and OSX, Part 1.
- Share Extensions talk by Guy Fullerton.
  - Usually view controllers in addition to plist.
  - CFBundleDisplayName.
  - NSExtendion.
    - NSExtensionAttributes.
      - NSExtensionActivationRule.
- SLComposeServiceViewController.
- UIActivityViewController.
WWDC 2014 Creating Extensions for iOS and OSX, Part 2.
Want to a share URL to App Store? See Technical Q&A 1633.

San Francisco travel bookmarks

xenodium.com @alvaro — Wed, 18 Mar 2015 02:00:00 GMT

Istanbul travel bookmarks

xenodium.com @alvaro — Wed, 18 Mar 2015 02:00:00 GMT

Altan Şekerleme (turkish delight shop).
Cheap eats in Istanbul's Bazaar District.
Findikli rainbow stairs.
Historic neighborhood of Arnavutkoy in Istanbul.
Istanbul Food: after the perfect bite.
The rise of Karakoy: Istanbul's hippest neighbourhood.
Uskudar Fish Market.

Rome travel bookmarks

xenodium.com @alvaro — Wed, 18 Mar 2015 02:00:00 GMT

Adaptive Sampling

Code & Visuals — Wed, 18 Mar 2015 02:00:00 GMT

Adaptive sampling is a relatively small and simple but very powerful feature, so I thought I’d write briefly about how adaptive sampling works in Takua a0.5. Before diving into the details though, I’ll start with a picture. The scene I’ll be using for comparisons in this post is a globe of the Earth, made of a polished ground glass with reflective metal insets for the landmasses and with a rough scratched metal stand. The globe is on a white backdrop and is lit by two off-camera area lights. The following render is the fully converged reference baseline for everything else in the post, rendered using VCM:

As mentioned before, in pathtracing based renderers, we solve the path integral through Monte Carlo sampling, which gives us an estimate of the total integral per sample thrown. As we throw more and more samples at the scene, we get a better and better estimate of the total integral, which explains why pathtracing based integrators start out producing a noisy image but eventually converge to a nice, smooth image if enough rays are traced per pixel.

In a naive renderer, the number of samples traced per pixel is usually just a fixed number, equal for all pixels. However, not all parts of the image are necessarily equally difficult to sample; for example, in the globe scene, the backdrop should require fewer samples than the ground glass globe to converge, and the ground glass globe in turn should require fewer samples than the two caustics on the ground. This observation means that a fixed sampling strategy can potentially be quite wasteful. Instead, computation can be used much more efficiently if the sampling strategy can adapt and drive more samples towards pixels that require more work to converge, while driving fewer samples towards pixels that have already converged mid-render. Such a sample can also be used to automatically stop the renderer once the sampler has detected that the entire render has converged, without needing user guesswork for how many samples to use.

The following image is the same globe scene as above, but limited to 5120 samples per pixel using bidirectional pathtracing and a fixed sampler. Note that most of the image is reasonable converged, but there is still noise visible in the caustics:

Since it may be difficult to see the difference between this image and the baseline image on smaller screens, here is a close-up crop of the same caustic area between the two images:

The difficult part of implementing an adaptive sampler is, of course, figuring out a metric for convergence. The PBRT book presents a very simple adaptive sampling strategy on page 388 of the 2nd edition: for each pixel, generate some minimum number of initial samples and record the radiances returned by each initial sample. Then, take the average of the luminances of the returned radiances, and compute the contrast between each initial sample’s radiance and the average luminance. If any initial sample has a contrast from the average luminance above some threshold (say, 0.5), generate more samples for the pixel up until some maximum number of samples per pixel is reached. If all of the initial samples have contrasts below the threshold, then the sampler can mark the pixel as finished and move onto the next pixel. The idea behind this strategy is to try to eliminate fireflies, since fireflies result from statistically improbably samples that are significantly above the true value of the pixel.

The PBRT adaptive sampler works decently, but has a number of shortcomings. First, the need to draw a large number of samples per pixel simultaneously makes this approach less than ideal for progressive rendering; while well suited to a bucketed renderer, a progressive renderer prefers to draw a small number of samples per pixel per iteration, and return to each pixel to draw more samples in subsequent iterations. In theory, the PBRT adaptive sampler could be made to work with a progressive renderer if sample information was stored from each iteration until enough samples were accumulated to run an adaptive sampling check, but this approach would require storing a lot of extra information. Second, while the PBRT approach can guarantee some degree of per-pixel variance minimization, each pixel isn’t actually aware of what its neighbours look like, meaning that there still can be visual noise across the image. A better, global approach would have to take into account neighbouring pixel radiance values as a second check for whether or not a pixel is sufficiently sampled.

My first attempt at a global approach (the test scene in this post is a globe, but that pun was not intended) was to simply have the adaptive sampler check the contrast of each pixel with it’s immediate neighbours. Every N samples, the adaptive sampler would pull the accumulated radiances buffer and flag each pixel as unconverged if the pixel has a contrast greater than some threshold from at least one of its neighbours. Pixels marked unconverged are sampled for N more iterations, while pixels marked as converged are skipped for the next N iterations. After another N iterations, the adaptive sampler would go back and reflag every pixel, meaning that a pixel previously marked as converged could be reflagged as unconverged if its neighbours changed enormously. Generally N should be a rather large number (say, 128 samples per pixel), since doing convergence checks is meaningless if the image is too noisy at the time of the check.

Using this strategy, I got the following image, which was set to run for a maximum of 5120 samples per pixel but wound up averaging 4500 samples per pixel, or about a 12.1% reduction in samples needed:

At an initial glance, this looks pretty good! However, as soon as I examined where the actual samples went, I realized that this strategy doesn’t work. The following image is a heatmap showing where samples were driven, with brighter areas indicating more samples per pixel:

Generally, my per-pixel adaptive sampler did correctly identify the caustic areas as needing more samples, but a problem becomes apparent in the backdrop areas: the per-pixel adaptive sampler drove samples at clustered “chunks” evenly, but not evenly across different clusters. This behavior happens because while the per-pixel sampler is now taking into account variance across neighbours, it still doesn’t have any sort of global sense across the entire image! Instead, the sampler is finding localized pockets where variance seems even across pixels, but those pockets can be quite disconnected from further out areas. While the resultant render looks okay at a glance, clustered variance patterns becomes apparent if the image contrast is increased:

Interestingly, these artifacts are reminiscent of the artifacts that show up in not-fully-converged Metropolis Light Transport renders. This similarity makes sense, since in both cases they arise from uneven localized convergence.

The next approach that I tried is a more global approach adapted from Dammertz et al.’s paper, “A Hierarchical Automatic Stopping Condition for Monte Carlo Global Illumination”. For the sake of simplicity, I’ll refer to the approach in this paper as Dammertz for the rest of this post. Dammertz works by considering the variance across an entire block of pixels at once and flagging the entire block as converged or unconverged, allowing for much more global analysis. At the first variance check, the only block considered is the entire image as one enormous block; if the total variance e_b in the entire block is below a termination threshold e_t, the block is flagged as converged and no longer needs to be sampled further. If e_b is greater than e_t but still less than a splitting threshold e_s, then the block will be split into two non-overlapping child blocks for the next round of variance checking after N iterations have passed. At each variance check, this process is repeated for each block, meaning the image eventually becomes split into an ocean of smaller blocks. Blocks are kept inside of a simple unsorted list, require no relational information to each other, and are removed from the list once marked as converged, making the memory requirements very simple. Blocks are split along their major axis, with the exact split point chosen to keep error as equal as possible across the two sides of the split.

The actual variance metric used is also very straightforward; instead of trying to calculate an estimate of variance based on neighbouring pixels, Dammertz stores two framebuffers: one buffer I for all accumulated radiances so far, and a second buffer A for accumulated radiances from every other iteration. As the image approaches full convergence, the differences between I and A should shrink, so an estimation of variance can be found simply by comparing radiance values between I and A. The specific details and formulations can be found in section 2.1 of the paper.

I made a single modification to the paper’s algorithm: I added a lower bound to the block size. Instead of allowing blocks to split all the way to a single pixel, I stop splitting after a block reaches 64 pixels in a 8x8 square. I found that splitting down to single pixels could sometimes cause false positives in convergence flagging, leading to missed pixels similar to in the PBRT approach. Forcing blocks to stop splitting at 64 pixels means there is a chance of false negatives for convergence, but a small amount of unnecessary oversampling is preferable to undersampling.

Using this per-block adaptive sampler, I got the following image, which again is superficially extremely similar to the fixed sampler result. This render was also set to run for a maximum of 5120 samples, but wound up averaging just 2920 samples per pixel, or about a 42.9% reduction in samples needed:

The sample heatmap looks good too! The heatmap shows that the sampler correctly identified the caustic and highlight areas as needing more samples, and doesn’t have clustering issues in areas that needed fewer samples:

Boosting the image contrast shows that the image is free of local clustering artifacts and noise is even across the entire image, which is what we would expect:

Looking at the same 500% crop area as earlier, the adaptive per-block and fixed sampling renders are indistinguishable:

So with that, I think Dammertz works pretty well! Also, the computational and memory overhead required for the Dammertz approach is basically negligible relative to the actual rendering process. This approach is the one that is currently in Takua a0.5.

I actually have an additional adaptive sampling trick designed specifically for targeting fireflies. This additional trick works in conjunction with the Dammertz approach. However, this post is already much longer than I originally planned, so I’ll save that discussion for a later post. I’ll also be getting back to the PPM/VCM posts in my series of integrator posts shortly; I have not had much time to write on my blog since the vast majority of my time is currently focused on my thesis, but I’ll try to get something posted soon!

NotMuch is awesome

Erethon's Corner — Tue, 17 Mar 2015 08:18:56 GMT

In one of my previous posts, I explained my email setup in detail. Since then I've added one more piece of software on the "stack". It's called NotMuch and it's an email indexer.

Italy travel bookmarks

xenodium.com @alvaro — Tue, 17 Mar 2015 02:00:00 GMT

22 Towns in Italy That Are Almost Too Perfect Looking.
Abbazia di Monte Oliveto Maggiore.
Abbruzzo's Sulmona: one of the oldest towns in the area and home to the Latin poet Ovid, confetti (sugared almonds) and arrosticini (barbecued lamb skewers).
Alberobello - Wikipedia.
Amalfi coast's Furore beach.
Boboli Gardens, Florence.
Braie lake.
Caffe Meletti.
Cinque Terre.
Emilia Romagna. (foood!).
Fixe small towns in Italy.
Herculaneum - Wikipedia.
Italy's six best road trips.
Le Sirenuse (Positano).
Live the good life: 12 local experiences on the Amalfi Coast.
Manarola.
Milan - Deus cafe.
Milan - Gelato joints.
Milan - Il mangione to find restaurants.
Milan - Mercato Metropolitani.
Milan - Navigli and eat at "el brellin".
Milan - Taveggia for hot chocolate.
Milan - Where to go for aperitivo in Milan.
Montalcino, and Brunello wine.
Palace of Venaria.
Piazza dei Miracoli.
Piedmont.
Pienza, Tuscany.
Rabbit beach.
Re di Macchia, restaurant in Montalcino.
San Galgano, Tuscany.
San Gimignano, Tuscany.
San Lorenzo leather market.
Skiing in Italy: find your perfect resort.
The Devil's Bridge in Borgo a Mozzano, Tuscany, Italy - Love from Tuscany.
The New Treasures of Pompeii.
Volterra, Tuscany.

Emacs lisp debug on entry

xenodium.com @alvaro — Tue, 17 Mar 2015 02:00:00 GMT

Wanted to track down which package was enabling ido-mode on my behalf. debug-on-entry to the rescue. Pass the method name in question and you're good to go.

(debug-on-entry 'ido-mode)

When done, use cancel.

(cancel-debug-on-entry 'ido-mode)

Burma travel bookmarks

xenodium.com @alvaro — Tue, 17 Mar 2015 02:00:00 GMT

Bagan and Inle lake are the "touristy" areas.
Balloons over Bagan.
For Bagan, get bicycles.
Inle lake guided boat tour.
Ngapali beach.
Ngwe Saung beach.
Shwesandaw at sunset.
zyklusdiewelt's Myanmar's photos.

New Releases for Cerberus and Eve

Nicola Iarocci — Tue, 17 Mar 2015 02:00:00 GMT

Yesterday Cerberus 0.8.1 was released with a few little fixes, one of them being more a new feature than a fix really: sub-document fields can now be set as field dependencies by using a ‘dotted’ notation.

So, suppose we set the following validation schema:

schema = {
  'test_field': {
    'dependencies': [
      'a_dict.foo', 
      'a_dict.bar'
    ]
  },
  'a_dict': {
    'type': 'dict',
      'schema': {
        'foo': {'type': 'string'},
        'bar': {'type': 'string'}
      }
  }
}

Then, we can validate a document like this:

The Shadow of the Wind (The Cemetery of Forgotten Books, #1)

Apurva Shukla's RSS Feed — Mon, 16 Mar 2015 09:00:00 GMT

⭐ ⭐ ⭐ ⭐ ⭐ As soon as I flipped to the first page, I was enchanted. I have no words, because it is just so superbly written with an excellent…

Fancy git aliases and git cherryfetch

The Technical Blog of James on purpleidea.com — Mon, 16 Mar 2015 07:36:19 GMT

Here are two quick git tricks that I’ve added to my toolbox lately…

I wanted to create a git alias that takes in argv from the command, but in the middle of the command. Here’s the hack that I came up with for the [alias] section of my ~/.gitconfig:

[alias]

    # cherryfetch fetches a repo ($1) / branch ($2) and applies it rebased!
    # the && true at the end eats up the appended args
    cherryfetch = !git fetch "$1" "$2" && git cherry-pick HEAD..FETCH_HEAD && true

Explanation:

OS X Screencasts to animated GIF

xenodium.com @alvaro — Mon, 16 Mar 2015 02:00:00 GMT

Alex Dergachev has a great howto for generating animated GIF out of OS X Screencasts.
Of interest GIF Brewery.

Can secure voting exist on the Internet?

Welcome to Marco Fioretti's website! on Stop at Zona-M — Sun, 15 Mar 2015 18:10:00 GMT

Last week I was asked this question: “If relatively secure financial transactions can exist, relatively secure voting can exist on the internet, can it not?" My answer? No. Because…

Ted 2

🦉 olshansky 🦁 — Sun, 15 Mar 2015 18:04:02 GMT

Olshansky's review of Ted 2

The Stanford Prison Experiment

🦉 olshansky 🦁 — Sun, 15 Mar 2015 10:45:38 GMT

Olshansky's review of The Stanford Prison Experiment

Writing Xcode plugins

xenodium.com @alvaro — Sun, 15 Mar 2015 02:00:00 GMT

Xcode-Plugin-Template from Delisa Mason/kattrali.
- Ensure DVTPlugInCompatibilityUUIDs is in Info.plist.
- Get from:

defaults read \
    /Applications/Xcode.app/Contents/Info DVTPlugInCompatibilityUUID

Luis Solanos's Xcode-RuntimeHeaders.
Boris Bügling's Xcode-RuntimeHeaders fork.
Building the Xcode plugin snapshots.
Xcode Explorer enables you to explore Xcode events/notifications.
Creating an Xcode4 plugin.
Common Xcode plugin techniques.
XcodeExplorer to monitor Xcode internals.
Debugging your Xcode plugin.

Uninstalling Alcatraz from Xcode

xenodium.com @alvaro — Sun, 15 Mar 2015 02:00:00 GMT

$ rm -rf ~/Library/Application\ Support/Developer/Shared/Xcode/Plug-ins/Alcatraz.xcplugin
$ rm -rf ~/Library/Application\ Support/Alcatraz

ps. Removing all plugins:

$ rm -rf ~/Library/Application\ Support/Developer/Shared/Xcode/Plug-ins/*

Attention is a Resource

Nicola Iarocci — Sun, 15 Mar 2015 02:00:00 GMT

Today, the New York Times’ SundayReview features a great column by Matthew B. Crawford: The Cost of Paying Attention.

Attention is a resource; a person has only so much of it […] What if we saw attention in the same way that we saw air or water, as a valuable resource that we hold in common? Perhaps, if we could envision an “attentional commons,” then we could figure out how to protect it.

Continuous integration for the Linux Kernel - Built within Docker

smcleod.net — Sun, 15 Mar 2015 02:00:00 GMT

Linux Kernel CI for Debian

Github: sammcj/kernel-ci

Those of us using technologies such as Docker and BTRFS or simply trying to gain a performance edge on the competition have a lot to gain from the features and performance of recent Kernel updates (especially from 3.18 onwards).

‘Enterprise’ Linux distributions such as RHEL & variants are concerningly out of date when comes to the Kernel. Many people seem to have forgotten what Linux is… Linux IS the Kernel.

Amounts 1.0.3 Bugfix Update

Swift Fox Software LLC — Sat, 14 Mar 2015 18:23:45 GMT

It has been a while since I have updated this page! I just wanted to let people know that there was a change to sqlite in iOS 8.2 that has caused issues with Amounts. I’ve just submitted a small bugfix update to Apple that should hopefully go out next week. In the meantime I will... Read more »

The post Amounts 1.0.3 Bugfix Update first appeared on Swift Fox Software LLC.

Prefill Emacs swiper with current region

xenodium.com @alvaro — Sat, 14 Mar 2015 02:00:00 GMT

The new swiper Emacs package is proving to be a great alternative to helm-swoop. Here's how to prefill with current region:

(defun ar/prefilled-swiper ()
  "Pre-fill swiper input with region."
  (interactive)
  (if (region-active-p)
      (let ((region-text (buffer-substring (region-beginning)
                                           (region-end))))
        (swiper region-text))
    (swiper)))

(global-set-key (kbd "C-s")
                #'ar/prefilled-swiper)

Change macOS app icon

xenodium.com @alvaro — Sat, 14 Mar 2015 02:00:00 GMT

Open the new icon (.icns) in Preview.
Click on large image
Select all (⌘-a).
Copy (⌘-c).
Ctrl-click on app icon.
Select Get Info.
Click on app icon (top-left).
Paste (⌘-v).
Done!

Chappie

🦉 olshansky 🦁 — Thu, 12 Mar 2015 18:28:43 GMT

Olshansky's review of Chappie

My email Server Setup

Erethon's Corner — Wed, 11 Mar 2015 20:52:37 GMT

Wanting to have more control over my email, I decided to setup my own mail server. I thought I had a pretty good understanding of how mail works, considering my mutt/offlineimap/imapfilter setup, but I must say I was pleasantly surprised while setting everything up.

Hack on Emacs London meetup bookmarks

xenodium.com @alvaro — Wed, 11 Mar 2015 02:00:00 GMT

European Lisp Symposium.
guide-key displays available key bindings.
iplayer-el Emacs interface to the BBC's iPlayer.
swankr REPL (swank protocol for R).

History Text Analysis Over Spreadsheets - A Poker Player and Developer's Road to Agile Project Management

Fred Wu (@fredwu) — Mon, 09 Mar 2015 23:50:23 GMT

Working with OS X and Emacs tips

xenodium.com @alvaro — Mon, 09 Mar 2015 02:00:00 GMT

From M-x all-things-emacs, Ryan McGeary's OS X/Emacs workflow.

Frequently used apps:

Dotfiles

Github.

Building ycmd

xenodium.com @alvaro — Mon, 09 Mar 2015 02:00:00 GMT

Build

id: build-3

$ git clone https://github.com/Valloric/ycmd.git
$ cd ycmd
$ git submodule update --init --recursive
$ ./build.sh --clang-completer

Test

$ python ycmd
  serving on http://127.0.0.1:54265

More info

Decoding Captcha's Presentation

Ben E. C. Boyter — Sun, 08 Mar 2015 22:41:13 GMT

A few days ago there was a lack of speakers for #SyPy which is the Sydney Python meet-up held most months and sponsored by Atlassian. I had previously put my hand up to help out if this situation ever came up and was mostly ready with a presentation about Decoding Captchas. I did not expect it to be so full that people were standing (largest crowd I had ever seen there). Thankfully it seemed to go over well and while I need to get more practice at public speaking I did enjoy it. A few choice tweets that came out of the end of the event,

Displaying Kindle clippings for the web

Ecce Homology on Bastian Grossenbacher Rieck's personal homepage — Sun, 08 Mar 2015 21:15:33 GMT

Since I got my Kindle, the number of quotes in my quote data base started to increase exponentially. Clicking “Highlight” to save an interesting passage is so tempting. Now, a few months later, I of course wanted some way to display this treasure trove. I basically wanted a version of the good old fortune command, but a bit more modern—for those Web 2.0 times we are living in.

Luckily, the clippings format is very straightforward and only involves a little bit of text parsing. Below you can find my first feeble attempt at displaying a random quote from my clippings file. To use the script, you first need your clippings file. You can usually find it under the root directory of wherever your Kindle is mounted, for example Kindle/documents/My Clippings.txt. Rename the file to Clippings.txt. Now place the script below and the clippings file in a folder that is being served by a web server that enables you to execute CGI scripts. And you are done.

Since the script is barely doing anything, I am releasing this version into the public domain. I would love to provide an example installation for the script but I am currently migrating between servers, so I do not want to dabble with any CGI scripts as well. I will write a follow-up post if the script becomes more polished.

Happy reading until then.

#!/usr/bin/env python3

import cgi
import random

def readClippings(filename):
  separator = "=========="
  blocks    = []

  with open(filename) as f:
    for line in f:
      line = line.rstrip()
      line = line.replace("\ufeff", "") # Replace broken BOM

      if line == separator:
        blocks.append([])
      else:
        blocks[-1].append(line)

  blocks.pop()
  return blocks

def splitClipping(clipping):
  publication = clipping[0]
  quote       = ""

  for line in clipping[2:]:
    quote = quote + line

  return publication, quote

if __name__ == "__main__":

  print("Content-type: text/html; charset=UTF-8")
  print()

  clippings = readClippings("Clippings.txt")
  clippings = [splitClipping(x) for x in clippings]
  clippings = [(x,y) for x,y in clippings if y]

  publication, quote = random.choice(clippings)

  print('''
<html>
<head>
  <title>Spark: A random quote from Kindle clippings</title>
</head>
<body>''')

  print('''
<p>
  <blockquote>
  %s
  </blockquote>  
</p>''' % cgi.escape(quote))

  print('''
<p style="text-align:right">
  &mdash; %s
</p>
''' % cgi.escape(publication))

  print('''
</body>
</html>''')

Regular bookmarks

xenodium.com @alvaro — Sun, 08 Mar 2015 02:00:00 GMT

Photography bookmarks

xenodium.com @alvaro — Sun, 08 Mar 2015 02:00:00 GMT

Paris travel bookmarks

xenodium.com @alvaro — Sun, 08 Mar 2015 02:00:00 GMT

Atelier Maitre Albert (known for its rotisserie chicken).
Au Passage. Small plates in fun atmosphere.
Berthillon Ice Cream.
Breizh Café (traditional gallete, savory buckwheat crepes).
Buvette Gastrotheque. Wine bar with small dishes.
Cheri Bibi, possibly hipster, underneath Sacre Coeur, good cocktails.
Creperie Josseline. Worth the queue. Drink Breton cider with crepes.
Holybelly Canal Saint Martin.
How To Spend a Culinary Weekend In Paris.
L'Aller Retour. The place for steak-frites.
L'Office. Wine bar/bistrot.
Le Barav'.
Le Petit Poucet (restaurant next to Place de Clichy, Paris).
Le Reminet (Paris restaurant).
Marché d'Aligre. Authentic neighbourhood market.
Marché des Enfants Rouge. Small covered market. Eat at food stalls.
Mosquee de Paris. City mosque. Drink mint tea at courtyard under olive trees or eat in restaurant inside. North African food.
Musee d'Orsay.
Paris: A Guide To Some Of The Best Cafes In Canal St. Martin.
Pierre Herme. For great pastries and macarons.
RATP for transport info including the "carnet" of 10 tickets.
Rodin Museum.
West Country Girl (crepes).
What are the best boulangeries and patisseries in Paris for each arrondissement? (Quora).
What are the best places to buy cheese in Paris? (Quora).
Where are best vintage stores in Paris? (Quora).
Where are the best flea markets in Paris? (Quora).

Org mode bookmarks

xenodium.com @alvaro — Sun, 08 Mar 2015 02:00:00 GMT

London food backlog

xenodium.com @alvaro — Sun, 08 Mar 2015 02:00:00 GMT

B_acklog

Saporitalia (Italian/pizza), Notting Hill.

Menus | Lunch, Dinner, Sunday Roast - Blacklock.

Food & Drink - The Barbary | Covent Garden Restaurant | Neal's Yard | London.

Pophams Bakery.

Yipin China | Restaurants in Islington, London.

Sunday (breakfast).

The Quince Tree Cafe London (Cafe), Paddington.

Bombay Bustle (Indian), Mayfair.

Kyseri (modern turkish), Fitzrovia.

Pied a Terre (fine dining), Fitzrovia.

The Ledbury (fine dining), Notting Hill.

Flour and Grape (Pasta), Bermondsey.

Honi Pokē (poke), Soho.

ZIA LUCIA (pizza), Highbury Islington.

Yeast Bakery.

Pavillion Cafe.

Palm Vaults.

Angel

[TODO]{.todo .TODO} Shawarma Bar - Berber and Q.

[TODO]{.todo .TODO} Bombay Burrito (Indian burrito takeaway & restaurant: Angel, Islington).

Baker Street

[DONE]{.done .DONE} L’ANTICA PIZZERIA DA MICHELE (Pizza), Baker Street.

Barbican

[TODO]{.todo .TODO} Sushi Tetsu (must book, well in advance).

Barnsbury

[TODO]{.todo .TODO} Sunday (breakfast, cafe, working).

Battersea

[TODO]{.todo .TODO} Tonkotsu (ramen).

Brick Lane

[TODO]{.todo .TODO} Chez Elles (french).

[TODO]{.todo .TODO} Fika (Swedish, cinnamon buns, coffee).

Bermondsey

[TODO]{.todo .TODO} Druid Street market

FAT London (Kimchee ).
& Cultured butter.

Bethnal Green

[DONE]{.done .DONE} Lahpet (Burmese) <2018-12-02 Sun> <2018-12-27 Thu>

id: lahpet-burmese-2018-12-02-sun-2018-12-27-thu

Brixton

[TODO]{.todo .TODO} Nanban (Japanese soul food).

Borough

[DONE]{.done .DONE} Padella (great fresh pasta).

[TODO]{.todo .TODO} Roasting Plant: Fresh Roasted Coffee <2019-01-15 Tue>

Camberwell

[TODO]{.todo .TODO} Sun of Camberwell (Roast beef, Yorkshire, and sticky toffee pudding).

[TODO]{.todo .TODO} Silk Road (Chinese, Lamb/cumin skewers).

Canada Water

[TODO]{.todo .TODO} Chuck burger (truck).

Cannon street

[TODO]{.todo .TODO} The Brigadiers (Indian) <2019-01-22 Tue>

Chiswick

[TODO]{.todo .TODO} The Coffee Traveller <2019-01-16 Wed>

Covent Garden

[TODO]{.todo .TODO} Punjab (Punjabi).

[TODO]{.todo .TODO} 鼎泰豐・Din Tai Fung・ディンタイフォン・딘타이펑 (soup dumplings) <2019-03-12 Tue>

[TODO]{.todo .TODO} Little Kolkata (London Restaurant - Kolkata Spirits) <2019-03-12 Tue>

[TODO]{.todo .TODO} Duende (modern Spanish).

[TODO]{.todo .TODO} Rock & Sole Plaice (fish and chips).

[TODO]{.todo .TODO} Timber Yard (coffee, working).

[TODO]{.todo .TODO} Chicks 'n' Sours.

[TODO]{.todo .TODO} Jidori (chicken yakitori) <2018-12-02 Sun>

[DONE]{.done .DONE} Dishoom (Indian), try Lamb raan.

[DONE]{.done .DONE} Sagar (South Indian): Great food!

Clapham

[TODO]{.todo .TODO} Joe Public Pizza.

Crouch end

[TODO]{.todo .TODO} The Haberdashery (coffee, working).

[TODO]{.todo .TODO} Blend (coffee, working).

Ealing

[TODO]{.todo .TODO} Kiraku (Sushi).

[TODO]{.todo .TODO} Wa Cafe (Japanese Patiserie).

Earls court

[TODO]{.todo .TODO} MAM (Vietnamese) <2018-12-02 Sun>

[TODO]{.todo .TODO} Jollibee UK (filipino fried chicken/fast food) <2019-03-12 Tue>

Edware road

[DONE]{.done .DONE} GOGI (korean): Average. Also, they lied to me and apologized with £10 credit.

Euston

[TODO]{.todo .TODO} Roti King (Indian/Pakistani/Singaporean): 40, Doric Way, Euston, NW1 1LH.

Farringdon

[TODO]{.todo .TODO} Quality Chop House.

[TODO]{.todo .TODO} Daddy Donkey (Mexican).

[TODO]{.todo .TODO} J+A (coffe, working).

Fitzrovia

[TODO]{.todo .TODO} Indian YMCA (inexpensive Indian).

[TODO]{.todo .TODO} Pastificio al dente (Italian, fresh pasta) <2019-03-12 Tue>

[TODO]{.todo .TODO} House of Ho (Vietnamese).

Golders Green

[TODO]{.todo .TODO} Cafe Japan (Sushi).

Hackney

[TODO]{.todo .TODO} Pidgin.

[TODO]{.todo .TODO} DabbaDrop (Indian delivery subscription).

Hammersmith

[DONE]{.done .DONE} Indian Zing (Indian): It was OK (not great).

[TODO]{.todo .TODO} Dragon cat cafe (bubble tea and wheel cake). <2019-07-14 Sun>

Hamstead

[TODO]{.todo .TODO} Jin Kichi (Japanese).

Harrow

[TODO]{.todo .TODO} Mazar (Afghan).

Hatch End

[TODO]{.todo .TODO} Chuck Burger.

Holborn

[TODO]{.todo .TODO} Prufrock Café (coffee, working).

[TODO]{.todo .TODO} Good & Proper (coffee, tea, working).

[TODO]{.todo .TODO} Noble Rot (wine bar/food) <2019-01-22 Tue>

Holloway Road

[TODO]{.todo .TODO} Xi'an impression (Xi'an)

Islington

[TODO]{.todo .TODO} Delhi Grill (Indian), try chicken makhani and naan.

[TODO]{.todo .TODO} BabaBoom (the kebab makers) <2018-12-02 Sun>

[TODO]{.todo .TODO} Roots N1 (Indian).

[TODO]{.todo .TODO} The pig and butcher (sunday roast).

[TODO]{.todo .TODO} Busan BBQ (Korean meets American diner/burgers and fried chicken).

[TODO]{.todo .TODO} Smokehouse Islington.

[TODO]{.todo .TODO} Katsuke 100 (Japanese tea/cake room). <2019-07-14 Sun>

Kensington

[TODO]{.todo .TODO} Clarke's, try the burger.

Kensal Green

[TODO]{.todo .TODO} Centro Galego de Londres (Gallician).

Kentish town

[TODO]{.todo .TODO} The Fields Beneath (coffee, working).

King's Cross

[TODO]{.todo .TODO} Itadaki Zen.

[TODO]{.todo .TODO} German Gymnasium.

[DONE]{.done .DONE} Yeah! Burger at Star of Kings.

[TODO]{.todo .TODO} Wing Wing (Korean fried chicken)s.

[TODO]{.todo .TODO} Sambal Shiok known for laksa and satai burger. <2019-07-14 Sun>

Leicester square

Curry House Ichibanya UK Japanese Restaurant London WC2H <2019-03-12 Tue>.

Leytonstone

Singburi (Thai) <2019-05-07 Tue>.

Liverpool street

[TODO]{.todo .TODO} Gunpowder (Indian).

[TODO]{.todo .TODO} Cinnamon Kitchen (Indian).

London Bridge

[TODO]{.todo .TODO} Santo Remedio (Mexican).

Marylebone

[DONE]{.done .DONE} Fischer's (Austrian). Great atmosphere and schnitzel. Not cheap.

[TODO]{.todo .TODO} Nambutei (Sushi).

[TODO]{.todo .TODO} Lurra (Basque).

Mayfair

[TODO]{.todo .TODO} Mayfair chippy (Fish and chips).

[TODO]{.todo .TODO} Ikeda (Japanese).

[TODO]{.todo .TODO} THE ARAKI (Sushi) / pricey.

Mornington Crescent

[TODO]{.todo .TODO} Asakusa (Japanese).

Notting Hill

[TODO]{.todo .TODO} Blend (coffee, working).

[TODO]{.todo .TODO} The continental pantry.

[TODO]{.todo .TODO} MAM (Vietnamese) <2018-12-02 Sun>

Old Street

[TODO]{.todo .TODO} Sardine (French).

[TODO]{.todo .TODO} Sasuke (ramen).

Olympia

[DONE]{.done .DONE} Aborz (Iranian). <2018-12-27 Thu>

Oxford street

[TODO]{.todo .TODO} Roti Chai (Indian).

Peckham

[TODO]{.todo .TODO} Ganapati (South Indian).

[TODO]{.todo .TODO} The Begging Bowl (Thai) <2019-05-07 Tue>.

Piccadilly

[TODO]{.todo .TODO} Urban tea rooms (coffee/tea/brunch) <2019-03-12 Tue>

[TODO]{.todo .TODO} Machiya (Japanese comfort) <2018-12-02 Sun>

[TODO]{.todo .TODO} YORI-Korean Restaurant <2019-03-12 Tue>

[TODO]{.todo .TODO} Rochelle Canteen @ The Institute of Contemporary Arts (ICA) (Pie and ping Sunday special) <2019-02-12 Tue>

[TODO]{.todo .TODO} Dark Chocolate Coated Chocolate Pearl, 190g - Fortnum & Mason <2020-06-14 Sun>.

Putney

[TODO]{.todo .TODO} authentic Japanese restaurant in Putney - Tomoe, London Traveller Reviews - TripAdvisor.

Sheppherds bush

[TODO]{.todo .TODO} Caco & Co (Portuguese cafe). <2019-07-14 Sun>

[TODO]{.todo .TODO} Chop chop (Noodle bar). <2019-07-14 Sun>

Shoreditch

[TODO]{.todo .TODO} Mast Brothers chocolate makers.

[TODO]{.todo .TODO} Sagardi (basque).

[TODO]{.todo .TODO} Look mum no hands (cofee, working).

[TODO]{.todo .TODO} J+A (coffe, working).

[TODO]{.todo .TODO} Good & Proper (coffee, tea, working).

[TODO]{.todo .TODO} Smokestak (BBQ).

[TODO]{.todo .TODO} Pho Viet 68 (Banh mi).

Sloane Square

[TODO]{.todo .TODO} Rasoi (Indian).

Soho

[TODO]{.todo .TODO} Le Hanoi (Vietnamese) <2018-12-02 Sun>

[TODO]{.todo .TODO} Cay Tre (Vietnamese) <2018-12-02 Sun>

[TODO]{.todo .TODO} Viet Food (Vietnamese) <2018-12-02 Sun>

[TODO]{.todo .TODO} Tao Tao Ju (Dim sum) <2018-12-02 Sun>

[TODO]{.todo .TODO} Lokhandwala London (Top Indian Tapas Restaurant & Bar in London) <2019-01-15 Tue>

[TODO]{.todo .TODO} DUM biryani (Indian) <2019-01-15 Tue>

[TODO]{.todo .TODO} Pastaio (Italian, fresh pasta) <2019-01-15 Tue>.

[TODO]{.todo .TODO} Bun House (ie. pork buns).

[TODO]{.todo .TODO} Temper restaurant (BBQ/tapas).

[TODO]{.todo .TODO} Golden Union (fish bar).

[TODO]{.todo .TODO} Melt Room (Cheese toasties).

[TODO]{.todo .TODO} Shotgun (BBQ).

[TODO]{.todo .TODO} Smack Lobster (Lobster rolls).

[TODO]{.todo .TODO} Smoking Goat (Thai), highly recommended.

[TODO]{.todo .TODO} Atari Ya (Sushi).

[TODO]{.todo .TODO} Yumi Izakaya (Japanese).

[TODO]{.todo .TODO} Jugemu (Japanese).

[TODO]{.todo .TODO} Darjeeling Express (Indian).

[TODO]{.todo .TODO} Hoppers Dosas, Rice, Roast, Kothu & Arrack (Sri Lanka and Tamil Nadu).

[TODO]{.todo .TODO} Timber Yard (coffee, working).

[DONE]{.done .DONE} Pizza Pilgrims.

[DONE]{.done .DONE} Kiln (Thai), highly recommended.

[DONE]{.done .DONE} BAO (Bao buns! enough said).

[DONE]{.done .DONE} SAID (italian chocolate shop). Awesome hot chocolate.

Southhall

[DONE]{.done .DONE} Brilliant restaurant (healthier Indian). <2018-12-27 Thu>

id: brilliant-restaurant-healthier-indian.-2018-12-27-thu

Southbank

[TODO]{.todo .TODO} Tonkotsu (ramen).

South Kensington

[TODO]{.todo .TODO} Hour Glass (Pub restaurant).

Smithfield

[TODO]{.todo .TODO} Bird of Smithfield (Sheppherd's pie, ox cheek, cheesecake).

Spitafields

[DONE]{.done .DONE} Som Saa (Thai), highly recommended. <2018-12-27 Thu>

[TODO]{.todo .TODO} Lahpet (Burmese) <2018-12-02 Sun>

Strand

[TODO]{.todo .TODO} India Club (around for 50 years) <2019-05-21 Tue>

Stoke Newingtom

[TODO]{.todo .TODO} The Haberdashery (coffee, working).

Tottenham Court Road.

[DONE]{.done .DONE} Kanada-Ya (rammen): Not bad.

Tower Hill

[TODO]{.todo .TODO} M. Manze (pie and mash).

[TODO]{.todo .TODO} Maltby street market.

Tufnell Park

[TODO]{.todo .TODO} Monsoon (Indian), try lamb naga.

Turnham Green

[TODO]{.todo .TODO} Chief Coffee.

Victoria

[TODO]{.todo .TODO} Dominique Ansel Bakery (Bakery) <2019-01-22 Tue>

[TODO]{.todo .TODO} Dominique Ansel Bakery <2018-12-11 Tue>

[TODO]{.todo .TODO} Bleeker (burger) <2019-01-22 Tue>

Walthamstow

[TODO]{.todo .TODO} Grillstock BBQ.

Waterloo

[TODO]{.todo .TODO} The Laughing Gravy.

[TODO]{.todo .TODO} Zen China.

West Hamstead

[TODO]{.todo .TODO} Nautilus (fish and chips).

[TODO]{.todo .TODO} Mamacita (Mexican).

Whitechapel

[TODO]{.todo .TODO} Lahore Kebab House, try seekh kebabs with roti.

[TODO]{.todo .TODO} Sushinoen.

Wimbledon

[TODO]{.todo .TODO} Dalchini (Indian), try spicy cocunut fish curry.

Brian Lovin: Host of Design Details

Sam Solomon — Sun, 08 Mar 2015 02:00:00 GMT

Brian Lovin is the host of the Design Details podcast and is a product designer at Buffer. He also runs The Kollection, a music curation blog. Hey everybody.

In Defense of the Midwest

ntietz.com blog - technically a blog — Sun, 08 Mar 2015 02:00:00 GMT

As an undergraduate, I always imagined that I would someday move to the SF Bay Area to live in the heart of the software industry. With this in mind, in my final semester at Kent State, I joined a Silicon Valley startup as their third engineer¹. The staff at that time was split: one founder and one engineer were in Mountain View, CA; one founder and one engineer were in Ohio; and one engineer was remote. Nearly every month in the first year, I flew out to the Silicon Valley office to work with the engineers out there.

Since then, we have grown to have a technical staff of about 20 people. We are split pretty evenly between the Silicon Valley office and the Ohio office. I spend most of my time in the Ohio office, but I do commute to the Silicon Valley one occasionally.

Nearly every time I go out to California, my coworkers ask me the usual question: "so, when are you moving to California?" It seems like for people in the Valley, moving to California is such an obvious choice that it isn't even a question of if I'll move, but when. However, I truly love the Midwest and that I want to stay here for as long as I can. It's not for everyone, but it is for me and maybe it is for you.

It is affordable

In San Francisco, the median one-bedroom apartment costs $3,120 / month. In Kent, Ohio, a really nice one-bedroom apartment will cost you at most $1000 / month. Salaries are much higher in the Valley than in Ohio, but even a low salary in Ohio can get you a very nice apartment.

There is lots to do

Another argument that's used is that "Ohio is in the middle of nowhere" implying that there is nothing to do here and life is boring, surrounded by cornfields. On the contrary, there is actually a ton to do in Ohio. Here's a tiny sampling of what I like:

Cleveland gets all the Broadway shows once they go off Broadway, but at about half the cost
We have great music here, including the world-renowned Cleveland Jazz Orchestra, and tons of bands come through
We have some great sports teams (hi, OSU) and a ton of great sports fans (hi, Browns fans)
We have great food at very affordable prices

Actually, I haven't found anything I could do out in the Valley that I could not also do back in Ohio, except maybe get killer sushi.

Midwesterners are Great People

More than anything else, I love the people in the Midwest. Here's why:

Our people are incredibly polite and caring. Out here, people greet you on the sidewalk even if they don't know you. Neighbors will come help push your car out of the snowbank it got stuck in. Cars will let let you merge when they don't have to.
Our people are, well, scrappy: even though the Browns continue to lose, year after year, you will find no fans more loyal than the Browns fans². This attitude is carried through most things we do: even if you fail over and over, you just keep trying and hoping.

On balance, I haven't found nicer people than in the Midwest.

The Weather

Not many people would claim that Ohio's weather is great, but count me among them. Our winters are fairly harsh and cold, but they make you truly appreciate spring when it comes. All the non-winter seasons are really nice: spring is pleasant and life is blossoming around you; summer is warm and laid-back; and fall is brisk and beautiful, with the leaves all changing colors.

Great Universities

Despite popular opinion, the great universities aren't limited to the two coasts: we have UW-Madison, UIUC, Northwestern, and OSU, to name just a few. (Carnegie Mellon is also nearby, even though it isn't technically in the Midwest.)

The Pace of Life

On both coasts, the pace of life is really, really high: you just go, go, go and work constantly. If you go to a restaurant or coffee shop, people around you are probably all talking something work related, because people don't slow down very much.

In the Midwest, though, people take a much more relaxed pace. If you go to a coffee shop, you'll find people talking about real life things, not work. Maybe they're talking about a book they read in their free time!

This is one of the things I love most about the Midwest - people actually turn off work mode sometimes and go relax. I firmly believe that, even in spite of this, people are not less productive here than in the Valley, because even though we may put in fewer hours, those hours are more energetic and we are more recharged.

The Midwest is a beautiful place filled with beautiful people. Don't write it off just because it isn't the heart of Silicon Valley - there is still a lot of good stuff and good work being done in this part of the country. Come visit, stay for a while.

¹ My company is in stealth mode and has requested that we not talk about the company publicly at this time. When that changes, I might have more to say about the company. Note: I can talk to individuals one-on-one if anyone is curious.

² We have fans on every continent including Antarctica.

Buxton's Rule

Nemo's Home — Sun, 08 Mar 2015 02:00:00 GMT

I consider myself a UX enthusiast. I consider that term to aptly describe my interest in UX. As I’m deeply involved in many UX and design decisions, I try to be well read on design and UX principles. While reading a discussion about iPhone prototypes on HN in June ‘12, I came across this comment:

Goes to show what it takes to achieve excellence: lots of trial and error. Produce at least 3 alternatives for every design decision (Bill Buxton agrees).

— mstuherl

It sounded so basic, yet often I see designers trying to defend their first design, because it seems good enough to them. No good design is ever born at the first step. Just like any other process, it takes multiple iterations to perfect it.

I recently got in touch with Morgan (mstuherl on HN), and thanked him for his comment. Here’s what he said when I told him I wanted to dub it mstuherl’s rule:

Hah! My name’s Morgan, so you can call it Morgan’s Rule if you like, but it comes from Bill, so Buxton’s rule would be more appropriate. His book Sketching User Experience contains yet more wisdom!

So thats what I’m calling it:

BUXTON'S RULE
Produce at least 3 alternatives for every design decisions.

UX bookmarks

xenodium.com @alvaro — Sat, 07 Mar 2015 02:00:00 GMT

Recipes

xenodium.com @alvaro — Sat, 07 Mar 2015 02:00:00 GMT

Jeera rice (cumin rice)

Sizzle spices (40 seconds)

1 tablespoon oil
1 teaspoon cumin seeds
4 cloves
2 black cardamon pods
1 cinnamon stick

Sautee onion (2 mins or browned/opaque)

1 small yellow onion (chopped)

Sautee rice + salt (1 minute)

2 cups of basmati rice
1.2 teaspoon of salt

Boil, then partially cover and simmer (8 minutes or water gone)

4 cups water

Rest 5 minutes (covered)

Slow-cooked lamb

Preheat oven: 240°C (no fan) 220°C (fan).
Lamb face up in tray.
Cook for 30 mins (or brown).
Take lamb out.
Add to tray: broth, onions, rosemary.
Lamb face down (broth covers 1/3 or 1/4).
Cover with lid (or baking/parchment paper then 2 layers of foil).
Bake for 3.5 hours at 180°C (no fan).
Take out.
Turn lamb face up (over again). Check if liquid needs top-up.
Cook for 2 more hours.
Check if ready. Does meat fall off the bone with fork?
If not, keep for another 30 mins. Check again.
You are done ø/.

Tom Kha Gai soup

Chicken or Prawns
2 kaffir lime leaves
1 lemongrass stalk
1 1/2 cocunut milk
3/4 sliced fresh galaghal
1 1/2 chicken stock or water
1/2 cup mushrooms
3 1/2 tbsp sugar
1/2 cup of cilantro
1-4 thai chillies
1-2 tbsp chili oil
1 green onion

Veg-Fruit juice

Lime
Ginger
Apple
Chilly
Celery
Fig
Blueberries

Berry Hempster

Hemp milk
Hemp protein
Strawberry
Blueberry
Date

How to cook Beef Chow Fun.

Cavolo nero with anchovies, chilli and garlic.

Vietnamese Pork Lettuce Wraps (my own versions)

Pork Mince.
Chopped garlic.
Chopped chillies.
Chopped ginger.
4 tablespoons of soy sauce.
2 tablespoons of apple cider vinegar.
Sesame oil (for cooking mince).
Half tablespoon of coconut sugar.
Lettuces (for wrapping).

Music backlog

xenodium.com @alvaro — Sat, 07 Mar 2015 02:00:00 GMT

[TODO]{.todo .TODO} CatMeows: A Publicly-Available Dataset of Cat Vocalizations | Zenodo.

[TODO]{.todo .TODO} Pay What You Want (bandcamp).

[TODO]{.todo .TODO} Zontali on Twitter: "Name one of your most emotionally resonant songs, and I ….

[TODO]{.todo .TODO} Generative.fm.

[TODO]{.todo .TODO} Diminished Fifth.

[TODO]{.todo .TODO} Archive of Indian music.

[TODO]{.todo .TODO} Budhaditya Mukherjee.

[TODO]{.todo .TODO} Debashish Bhattcharya.

[TODO]{.todo .TODO} Halim Jafar Khan and his Disciples - Sitar Quintet - LP published in India in 1968.

[TODO]{.todo .TODO} Harjinderpal Singh.

[TODO]{.todo .TODO} Jayanthi Kumaresh.

[TODO]{.todo .TODO} Kayhan Kalhor.

[TODO]{.todo .TODO} Malaya Chalo.

[TODO]{.todo .TODO} Mehboob Nadeem.

[TODO]{.todo .TODO} Nirmalya Dey.

[TODO]{.todo .TODO} Zia Mohiuddin.

[TODO]{.todo .TODO} Music sites

Sam Altman on Machine Intelligence

Nicola Iarocci — Sat, 07 Mar 2015 02:00:00 GMT

Sam Altman has a great short series on the development of superhuman machine intelligence (SMI). Machine intelligence, Part 1 is on why machine intelligence is something that we should be afraid of while Part 2 is on what we should do about it. And then there’s Technology predictions, the perfect closer.

Finding Communities of Collaborators in Paul Graham's Essays

Avy Faingezicht — Sat, 07 Mar 2015 02:00:00 GMT

In the world of startups, few people have a presence as strong as Paul Graham's. An avid computer scientist turned venture capitalist, pg (as he is known on the news aggregator Hacker News) is also famous for his thoughtful essays, which he publishes on a regular basis on http://www.paulgraham.com, and which spawn a wide range of topics. Almost every single article ever posted by Graham ends with a thank you note to those who discussed the topic with him, read his drafts, or somehow influenced him to write about the subject. Individually, these are just polite acknowledgements, but in aggregate, they offer a window into Graham’s personal network. The underlying assumption I will make here is that when Paul asks two people to read or review a draft of one of his essays, there is an implied relationship between the two collaborators' expertise, and this should be somehow encoded in the data. In this post I will try to analyze this system, and in the process explain how one can infer community groupings from the link structure in such a collaboration network. (TL;DR, here is the full image ) ### Getting the data To start the analysis, I wrote a quick python script that scrapes all the articles posted on the site, searched for strings starting with "Thanks to " and extracted the names that followed. This quick heuristic outputs a csv file where each row represents an essay, including its title and the group of collaborators listed. Naturally, the scraper made mistakes. Since Graham's writing doesn’t always follow the exact same format, some manual clean up was required, too. For example, names and nicknames had to be joined by hand (i.e., I had to tell the program that Mike and Michael Arrington are indeed the same guy). Armed with clean data, we can continue to the fun stuff. I will do my best to limit the math and the code, and show the concepts behind the algorithms instead. ### Lets build pg's network. We will start by building a network of essays and collaborators, and then use that network to produce what we are really looking for: a network that links collaborators to each other based on which essays they are co-listed in. The final goal is to find which of pg's collaborators tend to help him in similar essays, inferring "communities" of collaborators. Generally, networks are expressed as tables, or adjacency matrices, where rows and columns represent nodes. For the uninitiated: the words graph and network are used interchangeably to describe the system as a whole, while links and edges refer to connections between the nodes. To build the network, I created a node for each row in the essays data set, and as I iterated over the essays I added nodes and links for the contributors that correspond to each one. In pseudo-code, this means that: ```text For Essay in Essays: Add Essay to the Network For Person in Essay.Contributors: Unless Person already in Network: Add Person to the Network Add link(from: Person, to: Essay) to the Network ``` And that’s it! Now the network links all of the essays to their contributors, but there is one more step, since we want to link contributors to each other. Here is where the dual nature of the network kicks in to help. ### Bipartite matrices and projections By definition, the network we are studying is an unweighted, undirected bipartite graph; nodes are either essays or people. A bipartite network is a graph whose nodes can be grouped into two disjoint sets. In this case, essays, which only link to people, and people which only link to essays. We will exploit a property of this kind of graph that allows us to easily relate pairs of collaborators through the essays they have in common, by *projecting* the network onto the set of collaborators. This can be explained with some fancy math, but basically we count the number of times a pair of collaborators appear together, and we produce a new matrix that bakes this information into the importance of its links. ### Ok, done with the math. Back to pg. Using a python library called Networkx we can quickly compute pg's W matrix, which we then can plug into Gephi, and run its layout algorithm to get this: ![structure](https://cdn.faingezicht.com/pg-essays/structure.png) We're getting there, but the picture we have so far is not very useful, and the network structure is not telling us anything yet. ### Well, a bit more math... Remember that the end goal was to find subsets of people who help Paul with his essays before they are published, grouping them by which essays they appear in. We'll need some math to get these groups sorted out. To accomplish this, we will try to assign groups in such a way that we maximize the graph's modularity. I won't explain the metric in detail, but basically it amounts to how interconnected the members of each group are to each other. A high modularity shows that there are significantly more links between individuals of the same groups than when compared to a scenario where a node’s links are placed randomly across the network. Gephi maximizes modularity for us at the press of a button, using a complicated formula. I don't find this too intuitive, so I'll show a different way of solving this problem through a randomized process, even if it is not as efficient. We start at time t=0 with each node assigned to its own group, and compute the current modularity Q₀. We randomly select a node N, and we randomly update its group. We calculate modularity again, and if Q₁ > Q₀, we permanently set N's group to its new value. Otherwise, we maintain the previous groupings. This process is repeated a predetermined number of times T, eventually maximizing the value of Q. In pseudo-code, this is basically: ```text Assign each node to its own group Compute modularity Q_0 Select number of timesteps T For t in T: Select random node N Select random group G Assign N to G Compute modularity Q_t If Q_t-1 > Q_t: Reset N's group to its previous group ``` We let the algorithm run for a while, and we get groups! Since this algorithm is in the family of hill climbing heuristics, the clusters we find are not necessarily optimal, but only an approximation. Even the final number of groups is random, and can change on each run of the algorithm, but for our purposes this is good enough. ### The Graph To better understand the graph, we will give each group its own color. Furthermore, we will resize each node in relation to how many other nodes its connected to, a metric known as degree centrality. On this run, I ended up with 4 groups. When running clustering algorithms, it is always hard to tell what each group represents, but here are my guesses: #### Red: Investors With plenty of Y Combinator partners, plus Fred Wilson, Marc Andreesen, Ben Horowitz, Chris Dixon, Mike Moritz, among others, this one seems to be pretty clear. ![vc](https://cdn.faingezicht.com/pg-essays/VC.png) #### Teal: Startup Founders This one is a little bit trickier. Joe Gebbia, Kevin Hale, John Collison, Jason Freedman, etc. Most people in this cluster are founders of YC startups, but some of them have transitioned and now are part of YC as well. Somehow, Geoff Ralston and Ron Conway trickled in here, but they are clearly closer to the red group. ![founders](https://cdn.faingezicht.com/pg-essays/founders.png) #### Purple: Early Collaborators/CS Academics The content of pg's essays have changed over time, and the purple cluster seems to be picking up on his early collaborators: Jessica Livingston, Robert Morris, Trevor Blackwell, and Dan Giffin have been reading Paul's essays all along, and that brings them together in this group. Even his dad is included! Nevertheless, with people like Guido van Rossum, Matz, Steven Wolfram, Peter Norvig, and "several anonymous CS professors," this group definitely has a strong computer science flavor. ![early](https://cdn.faingezicht.com/pg-essays/early.png) #### Green: The blip And here, the experiment breaks down. Most people in this group have not really helped pg in many of his essays, and they are not very well connected to others in the graph. Maybe if we ran the modularity algorithm again this group would not be present, and its nodes absorbed by other. ![random](https://cdn.faingezicht.com/pg-essays/random.png) ### Update: removing nodes that appear once After posting my article, and tweeting about it, I got an email from pg saying that at a glance the groupings were ~70% right. Not bad for an unsupervised algorithm! He suggested that I should remove anyone who has only collaborated with him once, and this is the graph that popped out when I did that: ![redux](https://cdn.faingezicht.com/pg-essays/redux.svg) Click for larger size The fact that there are so few data points makes the modularity algorithm too conservative. In my opinion, this a more robust but less interesting visualization. ### Conclusion Networks are cool. By using this simple modeling tool, we got to pick up a lot of latent information, and by making simple additions we could improve our understanding even more. For example, we could produce the same analysis focusing on topic modeling for the essays, based on this clustering technique but projecting in the opposite dimension. Alternatively, we could also model the network dynamically, visualizing the network at each time step and seeing it evolve over time. This would allow us to see how the subjects pg focuses on change over time, from computer science and philosophy to business and startups. As it stands, this analysis is mostly a curiosity, but Paul's close relationship with his wife, Jessica, his protegé, Sam Altman, and his close business partners were all easily inferred with the degree centrality approach seen above. For better or for worse, Graham’s brilliant insights into the startup ecosystem (as influenced by the groups of people described above) have gained him an almost cult-like following. After this little project, I am thinking that maybe I am part of the cult too. Last but not least: Thanks to Hannah Doherty, Max Faingezicht, Jonathan Maduro, Leon Sasson and Paul Graham himself for reading drafts of this, and Mike Leonard for helping with the data cleanup. Also, thanks to Dirk Brockmann for getting me interested in Complex Networks. Most of the math and code above is derived from the notes I took in his course

UX scrapbook bookmarks