I’m Back

After months of hibernation, my blog is back up now hosted on an nginx server! Unfortunately a lot of my old posts haven’t been imported properly and I’ve lost all my attachments. I’ll be working on restoring that in the near future.

Focus

Some time ago, I wrote a post hypothesizing that “the cure for procrastination” is to become a busy person. With a constant stream of deadlines, the theory goes, one will not have the time to procrastinate.

I was wrong. It works, sure, but it’s the wrong thing to do. It forces you to schedule frivolous activities between the important ones. As Steve Jobs would say, if you want to do anything great, “cut the crap”. Focus on the one or two things that matter, and stop doing all the other crap that doesn’t matter. Jobs gave this advice to Larry Page, CEO of Google telling him to stop simultaneously working on hundreds of mediocre projects and instead focus on the ones that matter most. I’ve realized that this advice applies just as well to personal development. Doing hundreds of things at once will make it harder to become good at any one of them. Stop working on a hundred different projects and just work on them one at a time; Stop trying to please everyone and focus on the ones that matter most; Stop reading 6 different books at once and finish the one you’re reading before starting another one; Stop trying to learn eight different habits at once. Just focus.

When you negotiate with a friend, trust is the default

And when you negotiate with anyone else, distrust is the default. That’s what I’m starting to realize.

I suck at negotiating; never had to do it. As a software engineer in training, it’s just not something I’ve ever had to do in school and rarely in life. The one place where negotiation can happen a lot in life is when you’re bargaining for a better price e.g. on your insurance or an expensive purchase, but as someone who spent the majority of their life in the bubble of school, those situations never come up.

A conversation I had with a close software engineer friend went something like this:

me: Looks like I might be doing business with ABC
him: Don't forget to negotiate
me: Yeah, I suck at negotiating though
him: Yeah me too...that's a mistake I made during XYZ
both: *sigh*

I don’t know if this applies to many other engineering types, but I tend to have a diplomatic personality where if I respect someone enough to do business with, then I tend to lead towards trusting them. Often I would try and think about how to maximize the benefit to both of us rather than just myself. This is dangerous when the other person doesn’t behave the same way. It’s very easy to get short changed on a deal. When you trust someone, you’re more likely to agree with them. When you trust someone that you want a long term relationship with, it’s easy to let your guard down and let them take an upper hand while subconsciously thinking it’ll help build trust in the relationship, but if they’re experienced in negotiating, then you’re just a sucker who can’t negotiate.

Apply this to finding a co-founder and I can see why it’s good advise to start companies with your best friends rather than someone you’ve just met. When you talk with a friend, trust is the default. There are no incentives to deceive because the person you’re going to be working with is a friend–you’re happy when they’re happy and vice versa, so you both optimize for mutual benefit. The deal you agree to will often end up being the most fair regardless of each others negotiation skills. Without an initial trust, there’s a huge incentive to negotiate to maximize your own benefit, and that’s a game where the better negotiator wins. If you know you’re not the better negotiator, then you have to get better at it. Otherwise the only winning move is not to play because if either party feels shortchanged in this critical stage of the business relationship then you’ve already lost.

Visualizing Global Sales Data With Google Earth

This is a cross post from 22Michaels, a blog about the adventures of Michael Fox, Jodie Fox and Mike Knapp from Shoes of Prey.

As an early employee of the unique design your own shoe company Shoes of Prey, I’m intrigued by how quickly the company is growing. It seems like every time I walk into the office there is a new employee in Sydney or one of it’s partner offices around the world. As the business spreads into new markets across the globe, we start to ask questions like “how quickly are we growing in Japan?”, “how successful was our valentine’s day campaign?“, “how can we replicate our Australian success in Russia?”. We can take information like where and when we’re making a sale and apply basic statistics and number crunching-fu to try and answer some of those questions, but that’s no fun. What if we could see our growth with our eyes?

Google earth is an excellent tool for visualizing geo-location data, and that’s exactly what we chose to use. In a few hours, I was able to create a visualisation of shoe sales around the world over time. Here is a video of Australian sales:

Seeing the growth of Shoes of Prey over time is exciting, and it only takes the right tools and bit of programming background to put something like this together. This post gets technical from here, so skip to the end of that’s not your cup of tea.

Displaying time sequence animations in Google Earth is easy, it’s a matter of getting your data in the right format. It uses a format called KML, for which you can find documentation here. I’ll summarize what it took to put the animation together.

You can place a “placemark” on the map with the following tag:

<?xml version=”1.0″ encoding=”UTF-8″?>

<kml xmlns="http://www.opengis.net/kml/2.2">
  <Placemark>
    <Point>
      <coordinates>-122.0822035425683,37.42228990140251,0</coordinates>
    </Point>
  </Placemark>
</kml>

Furthermore, you can add a custom icon as well as a “TimeSpan” to specify when to display that “placemark”. This is what we use to display those red shoe icons on the map.

<?xml version="1.0" encoding="UTF-8"?>
<kml xmlns="http://www.opengis.net/kml/2.2">
  <Document>
    <Style id="iconStyle">
       <IconStyle>
          <scale>1.0</scale>
          <Icon><href>shoe.gif</href></Icon>
       </IconStyle>
   </Style>
  <Placemark>
      <TimeSpan>
        <begin>2009-10-09</begin>
        <end>2009-10-16</end>
      </TimeSpan>
     <styleUrl>#iconStyle</styleUrl>
     <Point>
       <coordinates>-122.0822035425683,37.42228990140251,0</coordinates>
     </Point>
  </Placemark>
</Document>
</kml>

The “kml” above will display a single shoe icon at GPS coordinates (-122.0822035425683, 37.42228990140251), you can add more points by adding more <Placemark> sections.

Creating and animating heat maps is more difficult and will require some programming. Python is a major programming language we use at Shoes of Prey and luckily I found a Python library that not only generates heat maps, but also generates kml code to overlay the heat map on Google Earth. The heat map library by itself had a few limitations for what we wanted to do; mainly, it was too slow and it was limited to only generating a single static heat map and kml file. However, the great thing about open source software with modification friendly licenses is that you can modify it if it doesn’t do exactly what you want it to do. I had to modify the library to make it faster and generate a time sequence of heat maps like the one you see in the video. You can find the source code of the modified version here. All you have to do is give it the data in the right format, the library does the hard work for you.

import heatmap
# Code to format your data goes here #
hm = heatmap.Heatmap()
hm.animated_heatmapKML(pointsets, outfile_name) 

The first argument to the animated_heatmapKML functino takes an array of 3-tupples in the format like below:

pointsets = [(start_date, end_date, [lat, lng]), (start_date, end_date, [lat, lng]), …]

The second argument is the name of the output file, e.g. “heatmap.kml”. There are other optional parameters, see the documentation for more details. That should generate a “heatmap.kml” file as well as associated heatmap images heatmap.kml1.png, heatmap.kml2.png, … to go with it. Open the file in google earth and slide the timeline to see the heatmap change over time.

What do you do with your business’ data and how do you use it to improve your business?

Internships and Grad Jobs Overseas

This is a cross post from Beta, a news letter for CSE students written by CSE students

If you’re a second year cse student or above, it might be time to start thinking about what you want to do after uni. A great way to get started is with an internship. Several companies offer internships in Sydney, but I hope to convince some of you that doing an internship overseas or even taking a grad job in the US is a good idea.

First, a bit of background: If you’re a good software developer who is about to graduate from a computer science (CS) or software engineering (SE) degree, your job opportunities in Australia are quite limited compared to a student with similar talents and skills in the US. Google is probably on top of your list, and then possibly Atlassian, and if you’ve done a bit of research, a few smaller companies that you like (and that’s a big if because they sure are hard to find. Ever heard of micro-forte? OKLabs?). Your other options include doing a PhD to further delay your entrance into the real world in the hopes that conditions will be better when you’re done, or self employment which includes being an indie or freelance developer and starting your own company; anything to make a decent living without ending up at a bank or any organization that doesn’t appreciate recursion or python’s clean syntax as much as you do.

Google, Apple, Microsoft, and Facebook are the big 4 of technology companies, and then there are some you might not have even considered, such as Amazon, Nintendo, Nvidia, and Adobe. Out of those companies, only Google hire software and CS grads in their Sydney office. If a PhD and self employment are off the table, then Google stands above the rest as the best place to work, whether as an intern while you’re doing your degree, or as a graduate when you finish. Google Sydney is a great place to work (just ask anybody currently working there); but they can’t hire everyone and for one reason or another, you may also not want to work in Google Sydney.

What you may not realize is that the big 4 tech companies (as well as many others like Amazon and Nvidia) all offer internships and grad jobs to students all over the world, including Australia. Since they’re all competing for the same pool of excellent CS students coming out of American universities, they treat their interns and employees very well, and this is reflected in their salaries and working conditions. These companies treat their interns very well. They will sort out your visas (even if you’re not an Australian citizen), fly you to the US, pay a decent salary, and subsidize a lot of your accommodation costs. A good CS or SE grad in the US can expect to get at least 75k USD as a starting salary plus benefits and bonuses, and often more. This figure was a lot better when the exchange rate was was favoring the USD, but it’s still a pretty good figure. A good CS student in the US will have done some internships at these companies and receive several offers from different companies, which make starting salaries even higher. In comparison, the same CS or SE grad in Australia will often have to work at a bank to get close to that salary, and the work conditions for software engineers will almost certainly be worse. If you’ve seen the recent cse email about starting salary statistics, the highest and many of the above-average salaries are paid to people going to work overseas.

I just returned from an internship at Microsoft, and overall it was a great experience. They had intern social events every two weeks and many of us got heavily subsidized rental cars to drive everywhere. Interns even got free gym membership like full time employees.

At Microsoft, the work environment is great for engineers. Most engineers get to work at around 9:30-10am, and leave at 5-6pm, most full time employees have their own office to minimize distractions, and an unlimited supply of free soft drinks, milk and coffee.

Google of course is famous for their free buffets, gaming rooms (and even laundry services in their Silicon Valley office), and I’m sure Apple and Facebook have their own perks as well. Mark Zuckerberg used to take interns out to lunch!

Even if your eventual goal is to have your own company, or even if you don’t want to move overseas, doing a 3 month internship in the U.S. at one of these companies is still a great experience, and unlike anything you’ll find in Australia.

Your peers might site Macquarie bank or Deloitte (two of the biggest recruiters of CSE students) as good options, but these companies are not technology companies. The best marketing student will want a job at Proctor and Gamble, not Microsoft’s marketing department. Similarly, if salary is equal, the best CS student will take a job at Google over a job at Macquarie’s IT department any day.

Of course, the focus of this article is about the benefits of working at big companies overseas. If you don’t want to move overseas, or prefer to work in smaller companies that move fast, allow you to use your favorite open source tools and decide which tools and frameworks to use that, look out for an article by me in the next few issues of Beta! I’ll be profiling some of the hidden gems of great places to work in Sydney that don’t quite have the recruiting budget of a big co to tell you about themselves.

 

Update:

I should also point out that the cost of living a fair bit lower in most places in the US is also lower than it is in Sydney, in addition, some states like Washington (where Microsoft headquarters reside) don’t have an income tax, hence you would only pay the federal income tax, which is much lower compared to the income tax in Australia.

Opportunity

As my friends and I walked out of a $15 dollar parking lot in an unfamiliar ally, a ruggedly dressed yet friendly looking man walked towards us and said, “y’know street parking here is free after 6pm, you can save yourselves some money by parking over there”, as he points to an empty parking spot.

We verbally thanked him for his advice and my driving friend proceeded to move the car while the man stood there and started a conversation with us. He then asked, “why don’t you give a poor man who helped you out some money for food?”

That’s when we realized that he wasn’t just any nice random stranger with no expectations of a return for his deed, nor was he just any random beggar. He was an opportunist, an entrepreneur in his own way. He saw the opportunity to save us money, presented the opportunity to us and negotiated a reward for his efforts.

We gave him a well deserved six dollars for his effort, more than the usual one or two dollars most beggars will manage to get from us.

Customer Service

Yesterday, I had my first haircut since coming to the US. Those who know me will know that my sense of fashion is about as good as a squirrel sense of nuclear physics. Needless to say, I went to the closest cheap barber in town. As I walked inside, the lady at the counter greeted me and asked for my number, name and address. Although we had just met, and she wasn’t particularly attractive, I reluctantly gave her my personal information. Why would a hair dresser need to know all that? I wondered, am I going to get telemarketing calls for the rest of my life? Is that how they keep the price so low? They’re not that sophisticated, the clerk couldn’t even spell my name (“Charles”). The clerk couldn’t hear any of my thoughts, so she typed up my details and told me to wait for the next available hair dresser.

By the time I got to a hair dresser, I was dying to know (pun intended), and almost as if she had read my thoughts, she explained “we record your number and address so we can write some notes about you, so the next time you visit we would know what your normal cut is, how often you come in and who your regular hair dresser is.” That was a big surprise. A little barber shop that serve hair cuts for $15 put that much effort into knowing their clients? They obviously try hard to have repeating customers, and that makes sense for a barber in a small town like Redmond. I don’t think any barber on the low price/style end in Sydney would do that, it makes me want to visit one in Seattle just to compare. The customer service here in places like restaurants and stores are much better than what I’ve experienced in Sydney, and this is just one of the examples.

Valuations Based on Partial Purchases are Almost Certainly ‘Overvalued’

There’s a lot of talk about Facebook and Groupon being valued at $70 billion and $15 billion respectively, and there is a common wisdom from booming real-estate industries that says that the best properties almost always look too expensive initially. But let’s get back to basics. The value of an item in a free market is defined as the price someone is willing to pay for that item. In the case of real estate, a house being worth $400k means that someone is going to pay (or take out a mortgage for) $400k to buy the house.

In valuating companies however, the valuation is entirely based on investment for a fraction of a company. That valuation is extrapolated based on the partial purchase. For example, if an investor decides to pay $1 million for 20% of a company, then that company is said to be “valued” at $1 million / 0.20 = $5 million. As someone with a basic understanding of economics would know, not everyone is willing to pay the same price for an item. Hence, the investor who invested $1 million for 20% of the company might be the person who is willing to pay the most for that company (presumably he’s the most excited about it), and he may not want to pay that price for the next 20%. In fact, it might be that no one is willing to pay more than $500k for 20%, and if everyone else only wants to pay $500k for 4 parts of 20%, then the total value of the company is worth $1 million + 4*$500k = $3 million, not $5 million.

The investor is of course looking for a return on that investment and would therefore expect that someone else is willing to pay a higher price in the future. This is usually based on growth. If a company is growing very fast, then it’s expected that the company will be worth a lot more in the future. Therefore, the premium price they paid today is based on the expected future price of the company, the present value is lower, much lower for a high growth company like Facebook.

I’m not saying that Facebook won’t be worth $70 billion some time in the future, but until you find enough people willing to pay that price for every share of the company, it’s not currently worth $70 billion.

Update: Here’s a list of companies worth less than the current facebook valuation: http://larrycheng.com/2011/01/12/the-companies-worth-less-than-facebook/, and a comment on HN that relates to what I’ve said here:

It’s important to note that stock valuations are derived from the expected future profitability of a company. Stock is a claim on future profits. Compared to a company like Halliburton, who has to build billion-dollar oil rigs before they make any profit, a technology company like Facebook is a lot more lucrative and thus its stock can inflate quickly if the profits expand faster than costs. And generally technology costs go down over time while profits increase exponentially. So you have a powder-keg scenario in Facebook if, and only if, they find a profitable business model based on their enormous userbase. And they better find that business model or the stock will crash like it’s the year 2000.

Compiling the latest mongodb in dreamhost VPS

This post has been sitting in my draft folder for a while, and it doesn’t look like it’s going to be finished any time soon, so here it is while it might still be useful:

I’ve been playing around with mongodb for a few days and as a hello world project, I decided to use it for the backend of an android app for Dinosaur comics. This was all fine on my dev machine, but when I tried to deploy it on my webhost, dreamhost, I got suggesting that the version of mongodb installed on dreamhost was too old. (The other piece of my backend was nodejs 2.5 and the mongodb driver for node, which seems to require the newer version of mongo). I went to the mongodb website and downloaded the pre-compiled binaries for mongodb, unfortunately they didn’t seem to work (it failed with core dump, incompatible architecture errors, and I haven’t investigated further than that). So I decided to try and compile from source.

To install the latest version of Mongodb (1.6.5 at the time of writing) on dreamhost VPS, first you need to install the dependencies:

These instructions apply to Debian 4.0 (etch), which is the current version of my VPS, yours may be higher. You can check which version of debian you’re using by typing

lsb_release -a

into the terminal.

The first thing I had to do was change the the package list to use a newer list of packages from what seems to be a defunct list. You might not have to do this if you’re using a more up to date version of debian.

Edit the package list in /etc/apt/sources.list to this:

deb http://archive.debian.org/debian etch main contrib non-free
deb http://debian.di.newdream.net/ etch ndn
deb http://archive.debian.org/debian-security etch/updates main contrib non-free
deb http://archive.debian.org/debian-volatile etch/volatile main contrib non-free
deb http://archive.debian.org/debian-backports etch-backports main contrib non-free
deb http://debian.dc-uoit.net/debian-multimedia/ etch main

Mongodb runs queries in javascript internally, so you’ll need a Javascript engine. Spider monkey will do. The version of spidermonkey in the debian packages were too old, so I had to compile my own version. You can build it by following these instructions:

building spider monkey
https://developer.mozilla.org/en/SpiderMonkey_Build_Documentation

spidermonkey install make file:
http://ebixio.com/blog/2010/07/31/how-to-install-libjs-spidermonkey/

Makefile for installing it once it’s built:

BLD := Linux_All_OPT.OBJ
#PREFIX := /usr/local
PREFIX := /usr

install:
cp ${BLD}/libjs.so ${PREFIX}/lib
cp ${BLD}/js ${PREFIX}/bin
cp ${BLD}/jscpucfg ${PREFIX}/bin
cp ${BLD}/jskwgen ${PREFIX}/bin
mkdir -p ${PREFIX}/include/js
cp *.h ${PREFIX}/include/js
cp *.tbl ${PREFIX}/include/js
cp ${BLD}/*.h ${PREFIX}/include/js

Now you can download and compile mongodb

wget http://downloads.mongodb.org/src/mongodb-src-r1.6.5.tar.gz

tar -xvvf mongodb-src-r1.6.5.tar.gz

cd mongodb-src-r1.6.5

./configure

make

make install

You might get an error like this:

g++ -o tools/sniffer.o -c -Wnon-virtual-dtor -fPIC -fno-strict-aliasing -ggdb -pthread -Wall -Wsign-compare -Wno-unknown-pragmas -Winvalid-pch -Werror -O3 -D_SCONS -DMONGO_EXPOSE_MACROS -DXP_UNIX -I. -I/home/chucknthem_admin/mongo tools/sniffer.cpp
tools/sniffer.cpp: In function ‘int main(int, char**)’:
tools/sniffer.cpp:503: error: invalid conversion from ‘const char*’ to ‘char*’

Fix it by doing this:

Go to tools/sniffer.cpp line 503 and change
pcap_bla(dev, blah)
to
pcap_bla(const_cast<char*>(dev), blah)

NOTES:

During compilation of either mongodb or spider monkey, you might run into some errors caused by missing libraries, I don’t remember exactly which ones I had to install, but feel free to ask me in the comments with the exact error you’re getting and I’ll try to help.

It seems to take a lot of RAM to compile Mongodb, so you might get an error like this:

g++: Internal error: Killed (program cc1plus)
Please submit a full bug report.
See for instructions.
For Debian GNU/Linux specific bug reporting instructions, see
.

That means that g++ was using too much memory and it was killed by the operating system. If you’re on a dreamhost VPS, then you can easily give yourself more RAM in the control panel (panel.dreamhost.com > VPS > Manage resources). I increased my RAM usage to 966mb up from 300mb and it compiled fine. So I would suggest the same.

Gripes with Android

As little as a month ago, if anyone asked me what phone to get, I would have told them to get an Android phone. Today, I’m not so sure. While I’m still a big Android fan, there are a few things missing or broken in Android that should have been there a long time ago.

1. Support for proxy settings for wireless networks. This wasn’t an issue until recently I started working for a company with a proxy authenticated wireless network. While there are ways to enable proxy settings on some rooted devices, they only work for simple proxy firewalls and doesn’t support authentication. While I work at a large software company with a mailing list of hundreds of self described Android enthusiasts, no one seems to have found a way to get on to the corporate wireless network consistently across devices.  This has to be the most complained about missing feature on the Android since release. My friends with iPhones and Win7 Phones do not share my pain as they both have proxy settings built in. If Android is going to take any market share away from Blackberry users, this has to be a high priority feature since many corporate wireless networks, and even many university wireless networks use proxy authentication. It’s such a basic feature for a wireless device that most people I’ve met, and myself included are surprised that it doesn’t exist when we have to use it.

2. Fragmentation of devices. This isn’t an issue with the majority of developers with CRUD apps that don’t have fancy graphics or hardware requirements, but many developers of games and widgets complain about the lack of standardization. While most Android devices are powerful enough for most developers needs, they still have to support the lowest end devices because they’re the ones whose users are going to leave one star ratings when they discover that the app doesn’t work. iPhone is the most consistent device to develop for and despite Android numbers overtaking iPhone numbers in the US, it’s still by far the most profitable. The recently launched Win7 Mobile has put some very high requirements for handset manufacturers intending to run Win7 Mobile. This will ensure that developers will be able to spend more time creating powerful high quality apps and less time worrying about device compatibility. There’s no doubt in my mind that this decision was made after seeing the problems with Android, and after seeing the device for myself and what others have thought of it, this is Microsoft’s shot at coming back into the smart phone market after the massive failure with the Kin phone.

3. Fragmentation of markets. This is not yet a huge problem, but every few weeks, I hear an announcement of some new Android market that is going to be launched promising to be better than what Google offers. In the US, phone carriers have began launching their own markets and some even won’t ship devices with the Android market built in. Amazon recently announced their developer preview of their Android market that will be much more like Apples Appstore with an approval process and quality standards for apps. Handster recently contacted many Android developers about their Market which from the looks of it might eventually support some new business models for apps. This is not strictly a bad thing, Google has said that “Android is an open platform” and people can do what ever they want with it, and no doubt there will be a lot of innovation from third parties trying different business models, and app distribution models to see what sticks, but this will hurt the users who will be overrun with choice about where to get apps. Apples innovation with the AppStore simplified how people bought software for their phone, they put everything in one place to reduce the amount of time looking for software, and put a quality standard in place to ensure that users where at least getting quality apps with a consistent design. Android seems to be doing the opposite.

4. The sorry excuse for a web interface for the android market. During Google IO 2010 in May this year, it was announced that there will be a Android Webstore like the iTunes store for the iPhone where users could browse and purchase apps and have those apps installed automatically on your phone. 7 months later, there is no word of this webstore. The closes thing we have is AppBrain, a third party app aggregator and app with a subset of the features of what the android web store should have been, including click to install on your phone.

5. Spam apps and the lack of quality requirements for Android apps. The android market is flooded with spam, junk and porn apps. Just take a look at the “Just in” section and you’ll see what I mean. The barrier to entry for publishing an app is almost non-existent. It’s completely inline with Google’s culture of giving users the power to decide what’s best, but it also detrimental to user experience.

I expect many of these features will be added or fixed over time, but it’s frustrating to see how slow things are moving. While a few months isn’t a long time, and Android certainly has improved a lot over the last few months, it’s still slower than the pace I would expect Google with the best Software Engineers in the world to go. Google’s fast moving startup days are disappearing and with so many products on the market, they’re just beginning to turn into a slow moving but very profitable tech company joining the ranks of Microsoft and IBM. I miss the days when the “don’t be evil” Google was young and quirky where everything they did was golden. And while they’re still the best company in the world, they’re losing their midas touch and as a Google fan, I’m starting to lose hope.