How To Compile Go code 40% faster with RAM disk

Go is already famous for it's impressive compilation speed. But if you came from a scripting language with practically no compile time, you're probably not satisfied. Here's the compile time after running go build main.go a few times to warm up the file system.

real 0m2.590s user 0m2.685s sys 0m0.775s

It's easily 2 to 3 times slower when the files aren't cached. Which could happen if your disk is experiencing a lot of usage. Here's the compile time when compiling from RAM disk. A whopping 40% faster; almost a second off.

real 0m1.871s user 0m2.124s sys 0m0.380s

Here's the bash script to get things working:

#!/bin/sh if [ ! -d ~/ramdisk ]; then mkdir ~/ramdisk fi sudo mount -t tmpfs -o size=512M tmpfs ~/ramdisk rsync -ah ~/go ~/ramdisk/ rsync -ah --exclude '.git' ~/path/to/project ~/ramdisk export GOPATH=$HOME/ramdisk/go

This creates a directory under the home folder as ~/ramdisk. Then assigns 512MB disk space and mounts it on the RAM. The rsync calls copy all Go files and project files to the RAM disk. Finally, it sets GOPATH to the new Go path under ~/ramdisk.

The next step is to reflect all file changes to the RAM disk instead of editing the files directly on it. This way you don't have to worry about losing your work. To do that we need a tool to watch for file changes and automatically duplicate the file. You can use any tool you like e.g. inotify, fswatch, nodemon etc. I'm going to use xnotify, a high level tool which can help with the build process.

./xnotify --verbose -i . --base /home/vagrant/ramdisk/project --batch 50 -- go build cmd/main.go -- ./main | xargs -L 1 ./copy.sh

copy.sh:

#!/bin/sh NAME=$2 SRC=/path/to/project/$NAME if [ -f $SRC ]; then echo Copying: $NAME cp $SRC ~/ramdisk/project/$NAME fi

The command above basically copies the file to the RAM disk and runs go build cmd/main.go && ./main when a file changes. Now if we want to stop using the RAM disk we just need to run this script:

#!/bin/sh sudo lsof -n ~/ramdisk sudo umount ~/ramdisk rm ~/ramdisk -r export GOPATH=$HOME/go
Keywords: go, golang, compile, speed, ram, disk, ramdisk

Theory of Cost Driven Testing

Purpose

In this article we shall explore guidelines that can help us reap the benefits of automated software testing with the least effort and time possible. There's always a trade off between writing bug free software and delivering results quickly. Thanks to the law of diminishing returns, we know that every additional effort produces less returns than before. This is especially true in startups and small dev teams with tight deadlines. Why are some software "bug free" despite having no automated test? Why do some software have many bugs despite high code coverage? Can we write tests that we are very likely to benefit from while not writing any test that is unlikely to help us?

Critical Features

Critical features are those that must be working correctly or there will be dire consequences. A bug in the feature could have catastrophic consequences such as losing your job, losing money, someone dying etc. This should always be the first to have automated testing. Thorough testing is highly recommended for expected and unexpected values since any bug could have potentially disastrous effect. Everything else can come later.

We have to be strict when deciding whether a feature is critical, or we might end up saying that 90% of the code is critical. Would a bug cause the app to crash? That's not critical. Would it make customer angry? That's not critical since it can be used as an excuse for every bug.

Bug Prone Code

Once all critical features are fully tested, we can move on to bug prone areas starting from the most likely to the least likely. Some of the factors that increase the chance of bugs are: amount of custom code, number inputs and outputs, logical complexity, high usage. That usually means testing high level functions such as controllers, instead of low level utility functions. This will let you get more code coverage with less tests, and also cover more flows, inputs and outputs. Functions which are made up of mostly third party party code will have lower priority since most of it are already be well tested. Features that are very frequently used should have higher priority simply because bug discovery is much more likely and more people will be affected by the bug. Compare this to a buggy feature that is barely used by anyone. There's a lower chance of discovering the bug. And even if the bug goes unnoticed, very few people would be affected by it. It is quite unnecessary to test functions that are simple and infrequently used since they are 1) less likely to be buggy, 2) less likely to be affected by code changes, 3) much easier to achieve 0 bug.

Since frequent usage increase the chance and impact of bug, it stands to reason that we should prioritize testing with data that users are more likely to enter. That means we should have lower priority for testing edge cases and bad input. Or simply not test them at all. For example, a user who enters his email in a date field might cause a horrifying error page, but he can easily rectify this by trying again and hopefully realizing this mistake. In the worst case, 1 user will have an unpleasant experience. Compare this to a user who enters the correct value into the correct field. There is simply no way to help himself. And everyone would be affected by it.

That's All

Since the goal is to write as little test as possible, any additional test means we waste more time and increase our technical debt. Hopefully with these guidelines we can write only 20% of the tests yet achieve 80% of the benefits of testing.

Keywords: unit testing, software testing, automated testing, programming

KPop Star Rain & Mediacorp Celebrities Red Carpet at Cartier - Ion

The Cartier flagship store at Ion Orchard has officially reopened and is now the largest boutique in South East Asia. There were special appearances by KPop star Rain and local celebrities Zoe Tay, Fann Wong, Christopher Lee, Rui En, Rebecca Lim and Sheila Sim.

Keywords: rain, mediacorp, celebrity, cartier, ion, opening, photography

Tiger Sugar Opening at Capitol Piazza

Popular Taichung brand Tiger Sugar has finally come to Singapore with the first store opening at Capitol Piazza. Creating the drink requires around 10 staff and 3 hours. The brand's emphasis on quality and authenticity extends to even the type of ice it uses. The shape of ice is similar to the ones typically drunk with whisky. The new store opened with a bang and a long queue that lasted over an hour.

Keywords: tiger sugar, opening, ceremony, event, lion dance, photographer

AIG Sentosa Golf Club and Tangling Rugby Club 2018

Some examples of official photographs from AIG Sentosa Golf Club and Tangling Rugby Club events held on 16 November and 17 November 2018.

Keywords: sports, youth, acitivity, photographer, event, photography, aig, golf, rubgy

Cosplay Convention Photos 2018

All photos were captured during Singapore Toy, Game & Comic Convention 2018. They were captured spontaneously while roaming through the crowds and while covering the events happening. Check out my other works to see what I can offer you.

Keywords: cosplay, event, photo, photgrapher, singapore, exhibition, stgxx

What Permission To Set For User File Upload Directory (Web Folder)

TLDR:

sudo chmod u=rwX,g=rwXs,o=rX -R path/to/folder
sudo chown www-data:www-data path/to/folder
sudo adduser username www-data

Introduction

If you've ever ran into a problem where your web app/server (Apache or NGINX) cannot read or write from a web folder (user upload directory), it's probably because of the file permissions. Unfortunately most people recommend setting it to 777, giving full access to everyone. Even some of the top answers on Stackoverflow recommend this. Let's see why this is a bad idea, and what's the correct way.

The Problem

To see why 777 is dangerous, we need to understand what's happening. It's giving read, write and execute access to owner, group and public! Let's imagine a common scenario. You have a PHP app that lets user upload to http://website.com/uploads/their-file. They upload a PHP file, virus.php, to http://website.com/uploads/virus.php. They visit that page… You're dead, because you've given execute access.

The Solution

u=rwX,g=rwXs,o=rX

I'm assuming you're familiar with chmod format. There are 2 letters which you might not be familiar with. X with a capital, means give execute access to directories, but not to files. This allows you to perform ls, but will not let you execute any script or program. s is a sticky bit that causes new directories inside to have the same owner. Useful when creating a new directory under a different user e.g. cron or ssh. In this case, we want new directories to belong to www-data group so that NGINX can access those files even if it was created by other users.

And of course remember to add your user to www-data group so that you will be able to access them through ssh or cron.

Keywords: permission, web, file, folder, directory, upload, linux, chmod, chown

Singapore Party Photographer For Hire

Have a launch party? Or maybe a birthday party? Or even a wedding anniversary or company party. You'll want to capture beautiful and fun moments to be remembered and shared. It takes patience and quick reaction to capture fun moments. That takes experience. Take a look at some of the events I've shot.

Keywords: party, launch, celebration, birthday, company, anniversary, photographer

How Much Do Singapore Freelance Charge (Photographer, EDesigner, Software Developer etc.)

Per Hour

Probably the most common way to charge a project is by the hour. Often the rate decreases as the hours increases. Usually to give a more competitive price and also because they save time. The downside to this method is that cost can exceed your budget if the time required increased. This is a bad approach for uncertain projects.

Additional Cost

Why do some projects cost more even if the duration is the same? Some projects require strenuous effort, more risk or addition cost to the freelancer. This cost would be factored into the rate they charge.

Minimum Cost

So you think you can hire someone for 10 minutes and pay much less? Tough luck! Some will reject without a beat, some might impose a minimum time, some will simply start at a much higher rate than they usually would when working for many hours.

Fixed Amount

Having a fixed amount both parties agreed on can be a benefit to everyone. Or not. When a fixed price is given, the freelancer would not charge more or less than stipulated. This means that if the project overruns the estimated time, the freelancer loses money. On the other hand, if the project completes early, the freelancer get all the profit. The advantage to this approach is the predictable cost.

Break Down Cost

Big project can be very expensive. Feel free to ask the freelancer for a break down of the cost to see where all the money is going. They should be able to provide this easily because they should have done the break down when estimating the cost.

Which Do I Choose?

Well, you don't really get to choose since each freelancer has their own preferred method. But if you don't like taking risk, go for the fixed amount.

Keywords: fee, rates, charge, singapore, freelance, photographer, designer, software developer, programmer