Sherif's Tech Blog

Just another guy on the Internet with a keyboard…

What Programming Language Should I Learn

So you want to start learning a programming language? The first question you might have is what language should you start with. Unlike when we’re born where we don’t get to pick the first language that we’ll start learning to speak, read, and write in – in the computer science world you have a choice. However, as a programmer you have a vast array of languages to chose from and sometimes I find people ask me “which is the easiest?”

The truth is whoever you ask will tell you their language is the easiest, best, most powerful, or whatever reason they can think of for you to learn and use that language! As a joke I posted this video on youtube along with others about individual languages, but this one seemed to get the most hits. Is it that people are very inclined to find the one ultimate programming language or just laugh about the rest? Who really knows, but it’s funny…

This is kind of like asking a multilingual person which language they think I’ll find the easiest to learn. While there may be some valid answers, most of them will probably be subjective and they won’t answer the true question one should be asking. That is, what am I going to be using the language for. Just like when you decide to learn French, because you’re either moving to France, or would like to communicate with someone who speaks French, similarly, you pick a programming language because you would like to communicate with a computer in a way that meets your objectives. What you intend to do with the language, and what it can do for you, however, may not always be so apparent at first.

I only recently realized how truly complicated it may be for someone who does not come from a programming or technical background to actually chose their first programming language. Having recently examined a massive comprehensive list of programming languages on wikipedia I found that there are currently over 600 programming languages to chose from and that doesn’t even include the more than 300 dialects of BASIC and some other various esoteric languages. This also probably doesn’t account for some of the lesser-known dialects or derivatives of some of these languages. Since not all programming languages have an official specification they very well may be implemented in dozens of different ways in smaller niches.

If we look at programming languages broken up into categorical or even chronological lists the information still doesn’t help make the choice any easier; or even more useful. However, if you simply attribute languages to their strongest generational origins you can narrow the list down to just a few dozen languages. If you take away some of the older generations and put emphasis on those languages highest in popularity and active use/development you come up with just a few languages and their respective dialects. However, this still isn’t informative enough to help someone decide their first language so I split up languages based on their strongest usage and in the world of computer science this comes down to two broad categories (systems programming and utility or application programming). The most notable distinction between these two types of programming is that systems programming aims to provide software to communicate with hardware while application programming usually aims to provide software for the user that’s sitting at the computer. So while software like a text editor or word processor is considered application software, software like a disk formatting/partitioning utility is considered systems software. Your operating system has to deal with the hardware in your computer directly in order to provide application programs with a means to do things like write to your computers memory while the operating systems page table and memory manager can control how this hardware is being used by the various application programs.

C

By far, still one of the most popular programming languages still around and even though it is still used by many to develop application software it hasn’t lost its popularity or its power.

C is not a language you usually pick to write every-day utility applications. If you chose to start learning C be prepared to start learning a lot of other systems programming concepts and technical hardware documentation as well. Most Computer Science majors take C as one of their first programming language courses in college. This is important, because there is a huge amount of software that’s written in C. For example, most operating system software is either written in C. There may be some C++ in there, but for the most part you’ll find a lot of linux distributions are made up of a huge amount of C code and much smaller portions written in either C++ or some other similar language. You may hear about Assembly language as well when learning or working with C. Essentially, when a C program is compiled into a native binary and run as an executable program it is technically transported to Assembler. You take a high-level language like C and, eventually, to get it to run on the machine it has to become low-level software in bytecodes the machine architecture can execute. C is still a high-level programming language, but it has also been recognized for its lack of agility in relation to programming languages like assembly which is a low-level programming language. Don’t let this confuse you, however, C is a powerful language and in fact many of the popular languages you will likely hear about or discover in this article were written in C. For example, PHP, Python, and Java are languages whose APIs and extensions were written in C.

However, C can be tough. Writing non-buggy C code is costly. It can take a lot of time, because you either have to find the libraries you need and implement them or write them yourself. C is a procedural and somewhat imperative language. It also teaches concurrent programming and programming with side-effects, which is very different from languages like Scheme where you program without side-effects. C programs are like one big global scope where everything can effect everything else. So you have to be very careful about managing your memory in C. You have to worry about pointers and references and data types everywhere in your code. You have the basic constructs like IFs, and loops, and functions, but ultimately you have to learn to do a lot of things other programming languages can make a lot easier, because they already have extensions that implement a lot of these popular C libraries built right into the language.

So, unless you plan on designing an API for a larger program or build some system utility C may not be the right language for you to learn. If you’re a compsci major you’re probably going to learn it as your first language whether you want to or not, but lets face it you chose the degree…

BASIC

BASIC has been around for quite a while as well and it has hundreds of dialects. It was popularized by many hobbyists during the 80s and grew further in popularity on Windows during the 90s with Microsoft’s Visual Basic suite that attempted to keep the language as simple but as powerful as possible. BASIC is not very difficult to learn, but it is also a compiled language like C and has declined in popularity over the last decade. It might not be the best language to work with, but it is still high on the hobbyists list. Much like languages that were once popular to learn just as a hobby and were fun to play with (like LOGO which was a dialect of LISP) not many people take it seriously.

BASIC has the essential control structures you’d find in almost any language like IFs, loops, and GOTOs, but it was fundamentally built on the concept of sequential programming where the entire program is built on one huge sequence of instructions. There are subroutines (like functions) and some dialects implement a lot of other modern features, but for the most part it’s great for when you want to learn programming for fun. If you’re serious about building cross-platform or enterprise-level applications BASIC is far from a first choice.

Java

Java and its other Java-based languages stand out for their compile-once run anywhere trait as opposed to many other compiled languages where you write the code once and then have to compile it for each different platform you chose to run it on. With Java, if you chose to compile your code to Java bytecodes to run in the JVM you will only need to compile it once. The JVM (or the Java Virtual Machine) can pretty much run on any platform (Windows, Linux, MAC OS, etc…) and works with the systems hardware directly through its VM. This enables programmers to be able to compile their Java code on any machine just once and it will run on any other machine in virtually any platform without having to recompile for that specific platform. Java can also be run in JRE (or the Java Runtime Enviornment) so it works as an interpreted language as well. Java’s popularity hasn’t declined much over the years and it’s gained quite the reputation with later adopting open source initiatives.

Java is also popularly taught in compsci courses in colleges, institutes, and universities around the world. It’s similar to C in that it is a statically typed language and has functions and basic loops and other constructs. However, Java is an object-oriented language. C is pretty much procedural in paradigm. You can build structs and things in C, but Java makes abstraction a whole lot easier with its OOP features. You can get a whole lot more done in development in a fraction of the time it might take you to do the same in C. So developing day-to-day applications in Java is a lot more common than with C. It’s just that a lot of the folks that have learned C and know it well have stuck to it over the decades and continue using it. Java is a much newer language. It appeared around the mid 90′s, but it has proven itself in the last 16 years or so. C has been around since the early 70′s and hasn’t changed much. The most current standard of C is C11, its predecessor was C99. Java is at Standard Edition 7.

Java is also considered a fast and secure language for a number of reasons. It is skeptical whether or not all of these reasons hold true, but for the most part they’re built on some solid grounds. First, Java code runs in the JVM, or the Java Vritual Machine, which means the VM can check the compiled bytecodes of the program and make sure they’re valid Java bytecodes before running or executing the code. Second, Java code is cross-platform so it easily translates to the same machine code across different platforms without much concern over the implemented libraries. Java is expected to be very performant because of its JVM. This means your Java programs run directly in a virtual machine that sits on top of the hardware layer allowing direct hardware implementations and interfaces as opposed to some other VM concepts where the program runs in the VM that runs on top of operating system or its implemented libraries that runs on top of the hardware. There’s somewhat of a more direct interaction there. Between Java and some other scripting languages like Perl, Python or PHP – this might be an advantage, but between C and Java it can go either way. In most cases C would easily out-perform Java, but in a few cases it might go the other way around.

PHP

PHP is probably the most popular language on the web. It has many followers and a huge open source community. It’s an interpreted language that was originally developed for producing dynamic web pages. However, today it is seen as a general purpose language. What makes PHP so great is that it works very well with web servers. You can install it as a web server module or run it on the command line. It has many useful built-in features that make web development easier right out of the box. PHP is also built on share-nothing architecture so it scales very easily and doesn’t require much configuration. It offers automatic memory management and it’s somewhat loosely typed so its data types may not be very suitable for edge cases, but that can be debated. For most general purposes PHP works great, but like BASIC it attracts a lot of hobbyists given that it lowers the bar of entry.

Unlike with C, in PHP you do not have to worry about managing your own memory. You can easily build data structures, facilitate external resources to databases or other libraries directly through the PHP extensions, and generate output to standard streams without a lot of fuss. It’s easy to take a general idea and implement it in PHP very quickly. Most people do this with Python and Perl as well to get a working prototype up and running. However, if you build a lot of prototypes, you know that they end up getting tossed out when you start building the real thing. Regardless, PHP is a great language to get code working quickly and very similar in syntax to languages like C and Perl. However, the down side is that these languages are also considered very ugly and have many extensions with poor implementations or interfaces or leaky memory. Not everything about PHP or Perl is great, but it works. At the end of the day it takes a fraction of the time to write PHP or Perl code that would do the same thing in languages like C and with less possibility of bugs since these languages are usually very forgiving and try to account for user error where possible.

PHP is extended by C and is built around the Zend Engine, which is the PHP Virtual Machine. PHP has different SAPIs, or Server APIs, for different web servers and platforms. Among the most popular are probably the Apache httpd module, which is known as mod_php and the fastcgi /fcgi SAPIs. The difference between the two is basically like running PHP inside your webserver as a part of the webserver program (mod_php), and running another program along-side of your webserver that interfaces with it through a CGI (Common Gateway Interface), which is what the cgi/fastcgi SAPIs are built around. There are lots of different implementations, but the module running as a part of the webserver usually trumps the others in performance and scale. PHP also has a CLI SAPI, which allows you to run PHP directly from the command line. You could use this to build command-line scripts like the popular BASH scripting language, on *nix shells. However, most people don’t use PHP to build command-line programs. It’s not the most performant programming language, but it works well for things like the web where you want to build dynamic websites or applications. Just tiny programs that execute for a very short period of time and run independently of one another. When you look into building things like long-running daemons, you usually turn-away from PHP and head for languages like C or even Java.

Other General Purpose Languages

There are many languages considered for both web development and as general purpose languages that are also dynamically or loosely typed and offer automatic memory management and even web server modules just like PHP. Languages like Python, Perl, and Ruby are also exceedingly popular and quite similar to PHP in many ways though they are not all based on the same generational languages. Of course shell scripting is also going to fit under general purpose in most cases and so Bash, sed, AWK, etc.. are also great languages to know.

To some people’s surprise, javascript is now becoming somewhat of a general purpose language itself. Recent VM implementations like Node.js make using javascript faster and a little more powerful than some of its earlier ancestors. One of the best things about javascript is it’s non-blocking nature and event-driven capabilities. It’s a great language for automating event-driven tasks by setting up listeners and such. It’s got a lot of uses on the web and offers multiple paradigms as well.

Beyond

Beyond just looking at what all of these programming languages can do for you it’s important to realize one language isn’t always enough to do what you need. If you’re going to start learning a programming language it’s easier to pick one that won’t require a lot of time to setup and configure. Something llike Python or PHP or even javascript is easy to just install and start writing code and the best part is you can just run that code instantly without having to compile anything and see the result right away. These languages aren’t very hard to learn because they have a lot of free online resources, documentation, and a lot of people already use them so you shouldn’t have too much trouble finding quick tutorials or examples of code that show you how to write short and useful programs. But of course your mileage may vary!

Over time, when you have learned your first programming language very well you may find the need to do some things that aren’t always very easy or even possible with that language (or you may never experience this depending on the language and what you’re doing). This may lead you to start using another language in place of or along side of that language for a similar project or a different project. If you’re a hobbyist doing this for fun you might not be so inclined to learn more languages, but if you’re a professional you will probably need to learn many languages over the years. It doesn’t hurt to have a long list of programming languages on your resume for a job and it certainly won’t hurt to already have some experience with a language you’ll be using on a new project at work. However, most programmers will be quite proficient in just two or three languages and have some overall understanding of others. This is usually all you need in the majority of cases.

Browsing the Web

When you browse the web today, compared to just 12 years ago, you find it’s a much more competitive market place. There wasn’t much commercialization taking place on the web in the mid to late 90’s. eBay and amazon were just about it; as far as consumer-based competition was concerned. The rest of the dot com start-ups were trying to make their money by reaching large audiences very quickly and profiting from that reach through advertisers — at the time being the most interested parties in said reach.

Some Things Never Change

Surprisingly enough though, some things haven’t changed much since. Many of these companies are still trying to gain a larger portion of the advertising market share through the web. When you consider some of the most popular websites on the web today, namely Google, facebook, yahoo, twitter, and probably still myspace, you realize all of these companies have something in common. They’re all trying to make a buck by getting the attention of parties interested in spending lots of ad dollars. Who are these advertisers? Pretty much any entity that is willing to pay money for exposing it’s advertisements to people.Google seems to have made the biggest stride in this area with more than 98% of it’s revenues coming directly from advertising over the last decade. Facebook won’t disclose any real financial data publicly, yet. So there’s no telling for sure if they are making any significant profits from their advertising revenues just yet. Yahoo at some point had seemed to be succeeding in gaining a significant portion of this market share, but apparently has bombed recently as its stocks have plummeted and it’s advertising business slowly starting to fall apart. As for twitter and myspace, they’re still both privately owned companies with little public financial disclosure, but – much like facebook – while they appear to be quite popular they don’t appear to have made any significant strides in advertising either (myspace still getting the bulk of its advertising from Google Adsense).

Looks Can Be Deceiving

Sometimes looks can be quite deceiving, though. For instance, when you consider some of the raw numbers that these companies claim – head held high – like the number of users they serve, or the number of people they employee, or the amount of revenues they generate, not all of it is put into clear perspective.

For eaxmple, facebook, recently claimed it had 800 million users. This is indeed a huge number of users. If you were to compare it to the average website on the web today or even the average corporation, this makes it seem like a huge company. But facebook only employs around 2,000 people and none of their users pay a single dime for using the service. The service is free so clearly 800 million people aren’t facebook’s customers. Then again what does facebook actually do for its 800 million users? Well, we know a lot about what they don’t do. They certainly don’t write all of the software that their users come back for so often. There are tens of thousands of facebook developers that either develop facebook apps or build on top of the facebook platform just for the benefit of having access to the huge social graph that gathers under the umbrella that is facebook. So nothing about their software actually makes their service unique per se, just that they have access to a huge userbase which interests a lot of parties. They also don’t do much of anything significant with what software they have built. If we consider their largest and probably most significant work it would have to be Casandra. HipHop might probably rank second. Their SDK is actually pretty much poorly documented and poorly supported, for the most part. Photos? Well, they do claim to have the largest photo sharing application on the web, but then again flickr isn’t far behind. And it’s not like facebook photos does anything really interesting with the actual photos apart from tagging (clearly their most popular feature). Other sites have actually done a far better job with photo sharing than facebook even though they may not be as popular. So in all reality these numbers – once put into some perspective – actually indicate pretty negative things about facebook even though it tries to put them in a positive light with a flashy spin (in hope that this will only boost their already growing popularity of course). The reality, however, is that this only indicates how under-staffed, under-paid, and over-commited facebook really is.

Here are some charts to demonstrate what these numbers actually indicate with some contrast for visualization.

Even if we consider every single user on facebook to be a customer of facebook, as a company, this would mean they are gaining the attention of about half the number of customers Microsoft currently attracts with it’s dominant Windows Operating system market share. They are neck and neck with Google in terms of users, but facebook claims to have surpassed Google in pageviews. There are probably very good reasons as to why that is. Let’s not forget people who visit facebook are usually their to do a lot of things. Chat with friends, browse every photo their friend ever took of their toe-nail, armpit, mirror-shot, bar-hoping-night, passed-out-magic-marker-art, and just about every other pointless moment their friends have ever captured on camera and uploaded to facebook for the world to see, not to mention the hundreds of wall-posts, messages, and I-Like hits the site gets every day from all of its users. In a single second facebook will probably have received around 2,000 photo uploads from its users. In about the time it has taken me to write this blog post they will probably have gotten around 3 Million new photo uploads. Each one of those uploads will generate a page view. Each time you play farmville or use an app that lets you know when your friend made a funny face or tried to some-how through the powers of dark magic poke you over TCP/IP that also generates what facebook considers a page-view.

The point I’m trying to make here is that facebook is clearly a content-provider. But just like Google, they aren’t the ones producing all of the content. However, unlike Google, facebook does actually facilitate a place for you to store and share all of this content and encourages it profusely by not setting a lot of stringent restrictions on how much content you can share.


If we took the amount of content users shared as an indication that a company was powerful in reach, however, we would have to say Hotmail is probably more of a social networking tool than facebook and twitter combined. Consider that Hotmail has only around 364 Million users and they share billions of emails every day – this would indicate more active sharing than facebook has ever openly claimed – even though Hotmail has less than half the userbase. Additionally if we compare the number of developers that actually work on supporting the service that these users are making use of facebook clearly has a higher developer to user load ratio than any of these other companies.

Stepping Outside of the Browser for a Minute

So it’s not clear that twitter, facebook, myspace or any of these other very popular social networking sites, are actually proving to be nearly as profitable as they are popular, despite them clearly being a great tool for people to communicate. The reasons for this are mainly the advertising risk. Twitter, facebook, myspace all clearly strong in numbers (any company that has to support a site that has users in the millions is nothing to laugh at), are still weak in strategy. There’s no good reason for a serious advertising (with billions of dollars to invest) to make long-term commitments to a user base that may very well disappear in a few years.

If you think of what Twitter is really all about, for example, it’s like reminds me of when we used to pass short little notes around in class during high school on tiny snippets of paper with messages like “see you at lunch…” or something silly like that. You basically send messages to people who choose to read them in tiny bite-sized pieces. This is nothing fascinating. The only reason the service is even popular is because a lot of people – at one point – found their friends using it (just  like myspace and facebook evolved) and decided to use it as well. That’s the thing about these social-based sites. They are easy to gather around and just as easy to turn away from. Eventually some one gets bored and a collapse of the social graph brings down the whole thing.

Google, on the other hand, doesn’t actually want to keep you within its borders all day long. It actually wants to get you the information you were looking for and out the door as quickly as possible with little to no distraction along the way. Quite the opposite from what facebook aims to do, because facebook understands it’s only value is in how long its users chose to stay. So they will do everything in their power to keep their users there longer. Google knows its users only come back because they know something else is out there to be found and that Google is probably going to find it for them more quickly (since it’s done so numerous times in the past with great success).

If you’re looking for a nice read on the subject of Google – by the way – I’d recommend this title right off my recent bookshelf Googled The End of the World as We Know It by Ken Auletta

Googled The End of the World as we Know It - Ken Auletta

Googled

It’s Not Browsing, It’s Searching

It makes sense that someone would turn to their friends or colleagues, etc, to find out about which of the latest blockbuster movies is popular when deciding on purhcasing a DVD or paying a visit to the movie theater, for example. But it might not make a whole lot of sense that someone would spend their time searching facebook when they’re looking for information on World War II history, or a photo of the point contact transistor at Bell Labs in 1947. People are probably more likely to turn to Google or Wikipedia for that type of search. Even though I have no doubt people are probably starting conversations on facebook groups or posting messages on people’s walls with very similar questions as you read this. To be fair, however, facebook and twitter probably do have good uses in searching for information as well. For example, you might hear about some new trend or some recent local, national, or even global event that just took place on one of these social networking sites, but it’s not like you can’t hear about that on the news either.

It’s just people’s naturally insatiable curiosity that won’t stop them from asking questions or looking for answers. This is why Google has become so popular today. They found the ultimate way to keep users coming back for more. Facebook, while it appears to have done the same thing, it apparently does it for all the wrong reasons. Google’s mission is to solve the problem of search (people are always asking questions about something) by making use of the worlds information as it becomes available (i.e. the web, books, news papers, television, etc…). However, facebook’s mission (while not even clear at this stage) seems to publicly say it’s attempting to do something fancy with something called the “social graph” (yes I really put a quote-unqoute on that). However, internally, facebook developers aren’t really sure what they’re doing. They seem to blog a lot about all these neat new technologies the company is getting its hands on, but what about this social graph? What does it do? How does it really make our lives better? Because we can communicate with our friends? No, that can’t be it. We’ve been communicating with our friends long before facebook ever showed up. Is it because facebook makes it easier to see what our friends are doing? No, that can’t be it either, because our friends still have to show us what they’re doing for us to see it (so instead of you seeing your friend getting drunk at the bar you get to see them in a photo on facebook drunk at the bar). Clearly, that isn’t making anyone’s life better (at least not yet). Is it because facebook offers a platform where you can connect millions of people and extract large quantities of useful information out of this so-called social graph? Well, to whose benefit is that, exactly? I don’t know about you, but I really don’t want everyone having access to all of my information just because I want to use their app to see what books my friends are reading (and yes there are very little to virtually no restrictions on how much information each app you use can access about you). I really don’t care to know how many people poke how many other people every day instead of getting off the computer or pulling their eyes away from their smart phone for a minute to take a look at the people in their immediate presence for a change. I don’t find anything about that useful, do you? Perhaps some might, but who? It’s probably not the average joe. It would probably be someone with a serious vested interest in you. Someone like say a company that produces hundred dollar designer jeans and seeks yet another way to pry into your personal details to figure out just how much influence they can muster to get you and all of your friends to buy those really expensive jeans that will just make your butt look spectacular.

At the end of the day, the numbers speak for themselves.

But, you’re just browsing the web. It’s not like any of this occurs to the average person on a regular basis as they “just browse the web”…

WebSockets – Making The Web More Useful

As of the date of this document (October, 2011) the WebSocket protocol has about 18 versions of the ietf-hybi draft. The hixie draft actually has around 71 Internet drafts since the first one in January of 2009. So in less than 3 years there must have been nearly a hundred drafts to this protocol with virtually every main-stream browser providing some different implementation of the spec. The earliest support offered for the WebSocket protocol in Chrome, Safari, FireFox, and Opera have actually been disabled by default in FireFox 4 and Opera 11 and Internet Explorer only has support through HTML5 Labs, which is just a prototype. Microsoft plans to offer better WebSocket support for future versions of IE with the new revised hybi-10 draft, IE10 is the planned release. If that weren’t enough the Gecko-based web browsers 6-7 implement the WebSocket API objects differently requiring developers to write extra code when integrating with existing WebSocket code. Still, it doesn’t even stop there… You now have to worry about the iOS mobile Safari browser (for iPhone) and the BlackBerry Browser in OS7 and how they support WebSockets as well. If it seems like this is all very messy it’s only because it is!
The web started as a place to make information more easily accessible by being able to do simple – but very logical – things like Hyper-Linking. Every document on the web should be able to link to any other document on the web regardless of domain or origin. That’s useful, because it makes information more accessible and efficient. However, today we spend the majority of our time trying to get automated scripts and tools to do most of the information processing for us so that we can ingest the processed information in bite-sized pieces that are easier to swallow. Much like how many of us have become accustomed to the processed meat we buy at the grocery store this is become rather second-nature. The only problem with that is the web was never designed to be this powerful. It doesn’t supply us with the right tools to do many of the jobs we expect it to do today. We simply never envisioned that one day the world would be running Software As a Service as we do today and probably wouldn’t have guessed that the World-Wide-Web was going to be the platform for all of this. The browser — once seen as a window into a strange and unfamiliar part of the Internet — has now become the corner-stone of the PC. We spend more time than ever before inside of our browsers doing many things we were once used to doing outside of the browser. Like playing games, chatting with friends, writing word documents, reading books, watching videos, editing and browsing photos, and much more. The world is now a much different place with the browser being at the center of every computer screen. Disconnect someone from the Internet and all-of-a-sudden their computer feels inert or even incomplete.
How the web works today and how it’s basically worked for the past 18 or 19 years is very simple. Your computer opens something called a TCP socket that connects to a remote server and on the other end the server is listening on a specific port for incoming requests. Once a request is received it processes the request according to a defined spec (called HTTP) and sends back a reply and the connection is terminated on both ends. There is no persistence layer. There are no negotiations to be made or handshakes to be exchanged between the client and the server. It’s all very simple and the protocol name says it all — it’s in plain text because it’s Hyper Text Transmission Protocol. The request may come from any client to a single server or from one client to multiple servers. It doesn’t matter. Each request is independent of every other request and every request must illicit a response. So there’s virtually no complexity in how this protocol can scale. We send off a request, wait for a response, and both the client and server part ways forever. That is, of course, until the next request is made.
To use WebSockets you need to write both a server and client. The client can easily be written in javascript using the WebSocket API (for WebSocket protocol capable browsers). The client can also be written using flash sockets as a fallback or even through long-polling if neither WebSockets or flash are available on the client machine. The server can pretty much be written in whatever server-side language you’d like. You can take your pick from any of the popular server-side languages like Python, PHP, Perl, Java, etc… Now you can even write one using javascript with node.js. I chose to write one using PHP and only implemented WebSockets (so there is no fall-back support for my example, but it works with most mainstream browsers like Chrome, FireFox, and Safari).

You can see a working WebSocket example here…

For some sample code on how to write a socket daemon in PHP you can see my PHP chat server example here as well.

This example basically uses WebSockets to implement a completely web-based chat in the browser. Each user that connects can send a message to the chat room and the message will be relayed to all other users currently in the chat — in real time. There is no database required on the server-side and no nasty long-polling done on the client. The TCP socket is a two-way pipe in full duplex that allows either party to send and receive messages at any time. The client doesn’t have to wait for a response to send the next request and the server doesn’t have to wait for a request to send the next response. It was easy enough to write the daemon in PHP and get it up and running quickly as an example, but you will need to review the specifications of the WebSocket protocol carefully and find a way to implement it that will work for your specific needs.

A useful full-blown framework you can use to help you implement most of this is socket.io but keep in mind if you’re going to be writing the server in a different language (not using node.js) you’ll want to understand the implementation well since it varies wildly and requires fall-back methods incase the client doesn’t support WebSockets.

Viral Videos and the Web

There’s a lot of power in viral videos on the web today. You can express an opinion or a thought and deliver to millions of people around the world with just a few minutes or even seconds of video. Youtube didn’t become popular because it did something revolutionary with video or because it developed any significant technology that made video better on the web during it’s early days. In fact, it grew too large too quickly for it to withstand the demand of its users and thus took on the deal with Google to maintain funding and the backing of a web startup that had the infrastructure necessary to expand their service. Youtube did, however, put the power of speech (or in this case video), back into the hands of the average person.

Television networks have been broadcasting what they see fit for decades before the web and the Internet ever came along. They do some market research, try to figure out what people want to watch and what forms of entertainment are most demanded and then they try to figure out a way to produce that content and broadcast it so that they can make a profit. There’s a key difference between that and what youtube did. Sure, there are plenty of users on youtube that will still try to submit some copyright-infringing video clip that some big-time production studio will try to get taken off, but there are also plenty of videos on youtube that are completely user-generated content. Like “Charilie Bit My Finger” as of today that video has gotten more than 377 million hits in just around four and a half years. That’s an average of 2 to 3 hits per second for four and a half years. So why are people clicking on Charlie Bit My Finger two or three times per second for years to watch two child and infant in a home video? For the same reasons millions of people watched America’s Funniest Home Videos on broadcast network television for decades. Except that it isn’t America’s Funniest Home Videos anymore and it isn’t owned by any network and there isn’ta  T.V Guide listing for when the show will air. There also isn’t the intervention of a studio editing your video. Individual users choose to share their own videos and the whole world decides for themselves if they’d like to watch.

Now, I’m going to assume that since hundreds of thousands of people have voted this video up over the years (or “liked” the video more than they “disliked” it) that the general audience finds this entertaining. But we didn’t have to pay an executive a six-figure salary and hire a marketing team to spend tens of thousands of man hours to figure this out in order to get there. It just works… Because people let it work.

So Viral Videos do have a significant impact on the web today. We apparently like watching other people or at least watching some creative expression of what they have to say. Whether that’s serious, comedic, or for any other various purposes. Sites like GoAnimate.com are pretty popular today. Allowing you to easily make and distribute your own animated videos and take your blog viral. I just wanted to include a brief demonstration of how easy it is to put your own videos on the web these days. It took me exactly 3 minutes to signup for a free account at GoAnimate.com, produce this video, and post it on my blog. But shooting your own home videos can also be just as easy.



GoAnimate.com: Facebook Changes Everything by GoogleGuy

Like it? Create your own at GoAnimate.com. It’s free and fun!

How To Build A Photo Sharing Application On The Web The Right Way

Personally, as a web developer, I’ve come across a number of clients that seem interested in doing some sort of web-based photo sharing application like flickr, or imgur, or photobucket. These are all pretty popular services on the web that allow you to share your photos with friends, family, colleagues, etc… They seem to be extremely popular and there’s no doubt that it’s evident people love sharing their photos online. Just take a look at facebook; possibly the world’s largest online photo sharing application that claims to get over 100 million uploads per day from it’s now more than 800 million users. With cameras found standard in such personal devices as phones, notebook computers, desktop computers, various other hand-held devices, and even (and don’t ask me why) TVs it’s no wonder we find it easy to store lots of digital photos and inevitably share them with others.

So I had to think long and hard about how I’d build an application or service like this so that it made good use of photos and made them easier to share and more accessible. The first thing that came to mind was checking out all the features these other services already had to offer and how they put them to good use. flickr allows you to do stuff like geotagging where you can tell people where the photo was taken. I remember either facebook or some app I may have tried a long time ago being able to do this as well. Since I log into my facebook account maybe two or three times a year I couldn’t say for sure, but what I do know is that facebook certainly got one thing right and that was not taking their users’ demand in wanting to share photos with friends for granted. When you’re able to point out who is in your photo that makes the information just that much more valuable and apprises the application for making photos more useful and accessible. There’s also a search value in tagging. But it can be pretty boring for a user to have to sit there and manually enter in all the information about each photo so we can rely on things like Exif where embedded information about the photo can be extracted by computers. You can get such information as GPS coordinates of where the photo was taken, a time stamp of when the photo was taken, the camera make and model that the photo was taken with, whether not the flash went off when the photo was taken, and even various other things such as focal length, exposure time, shutter speeds, etc… You can learn a little more about the Exif specifications here. However, keep in mind that not all cameras provide this format and not all of them are equipped to provide all of the different parts in the Exif header. Newer phones like the iPhone 4 are GPS capable and can embed GPS data into your photos if the GPS is turned on. There are also many digital cameras that either come with GPS devices embedded or can be purchased separately. Some other features you might want your application to have to make the process of having the user input information about each photo easier is facial recognition. This doesn’t have to be so sophisticated that it can automatically detect faces found in other pictures and tell you who’s who, but it can be helpful to let the software detect if a face exists in the photo and highlight it so that the user can simply type in who each person is for tagging purposes.

There are also some potential uses that I’ve found for OCR (or Optical Character Recognition) in photo sharing. If you can manage to extract enough significant textual data from the image you might be able to make certain aspects of search easier to locate photos. This is probably not going to be easy given that what little time I did spend fooling around with various OCR software proved that it has many setbacks and is clearly in the very early stages of development. Mostly OCR has trouble detecting text if the font size changes through the image, if the text encounters large skewed angles or if the text is rotated so that its orientation is not top-to-bottom and left-to-right. It’s also difficult to detect hand-written text, or text surrounded by other images, logos, or with too much depth or noise. There’s also quite a challenge trying to detect text written in various languages that don’t use the Latin alphabet. If it’s not a scanned image coming straight from the page of a book using a single evenly-spaced sans-serif font, with a fixed font size of around 12-20pt it will prove rather difficult to get any decent results from the OCR software.

So to give you an idea of what some good features of a photo sharing application might look like I put together a small working demo. My example makes use of most of the features I’ve discussed here, but leaves much to be desired, of course.

You can see the working Photo Application Demo here.

Here is a sample photo with embedded Exif information including GPS to demonstrate.

Here is a sample photo that demonstrates the applications ability to utilize some OCR techniques.

A Note About User Experience

OK, so I wanted to make sure this demo illustrates some of the basic functionalities a user might expect in a photo sharing app. The first thing you’ll notice is that uploading large photos and especially a lot of them can be a boring thing to do over HTTP. That’s because HTTP is built on a request/response model. You have to send a request first and then you eventually get a response back. Now if your request happens to be a 10 MB photo or even worse a 250 MB video (or even worse you don’t have high-speed Internet or your ISP offers lousy upload speeds) well… that’s a long wait with no indication at all to the user as to what’s happening on the other end. What’s even worse is if the user is uploading a file they aren’t aware is too large for you server to accept that can be a lot of waiting around for nothing. So I used some javascript combined with a script I found online at phpfileuploader.com and with a few modifications for security and improved user experience I hacked up a somewhat better interface for the user to deal with uploading multiple photos.

One thing you don’t want is to prohibit the user from being able to do anything else while they’re uploading their photos. Pop-up windows are just annoying, in my opinion at least, and forcing the user to wait until the entire upload is done before they can do anything else in the window is even worse. So if you try out my demo you can see that it doesn’t prevent you from browsing other photos on your computer to select for upload even while it’s doing the uploading simultaneously. You can even cancel an upload mid-way or cancel all pending uploads at once. It will also alert you ahead of time if you’ve selected too many files at once or if your images are too large or of the wrong file type. However, not all of these features are entirely reliable for security reasons as the user can easily bypass them. But they are their to improve the user experience. All the real security work is actually done on the server side. Even if you were to rename foobar.exe on your computer to foobar.gif and managed to upload it the server will still detect that it is an incorrect MIME type and reject the upload. This can be done relying on PHP’s File Info extension where it can check the file’s MIME type from the server’s supplied Magic MIME file.

Now, keep in mind that tracking the progress of the upload is just half of the battle. There’s also the part about viewing your photos after you’re doing uploading them. In my demo I used a javascript library called easybox which is based on the lightbox framework, but it plays nice with jQuery and seems to work a lot more smoothly in my opinion. You can download easybox from Google code and try it yourself. On the top right hand side of my demo where you see your gallery you can click on any of the thumbnails and it will use easybox to nicely let you view all of the photos in your gallery in a slide-show fashion without ever having to leave the page. You can also, of course, easily get a permanent link to each of your photos to share with others from the list of recently uploaded files below that. Your session will expire after 30 minutes of inactivity, however, and you will no longer be able to delete those photos. If you have cookies disabled you won’t be able to see what you’ve uploaded, but your photos will remain on the server indefinitely.

A Note About Using Javascript or Flash

One thing I wanted to point out was that your applications should definitely work better with javascript or flash or even Java Applets or whatever client-side components you may want to use to improve the functionality and user experience of your application. However, you should also consider that if your applications can’t work at all (if even with a degraded user experience or a limited use of features) then you’re not degrading your web applications in a graceful manner. In my demo, for example, I made sure even though I’m using a lot of javascript and even some flash to make the upload process a lot more user-friendly you can still use the application even without the javascript or if you don’t have flash installed. You can test this yourself and I even tested it on the Links web browser (which is a completely text-based browser with no javascript capabilities) and I could still manage to uplaod my photos and download them just fine with zero problems. This isn’t always possible, or easy, for every application, but it’s definitely a good idea to at least put in the effort to gracefully degrade your applications so that they can be somewhat usable to browsers of lesser capabilities. I even managed to get it to work on my phone (a Samsung) which is a pretty horrible phone with an even more horrible browser, but hey having a bad user experience is still better than having no experience at all.

A Note About Links

So I wanted to emphasize a little on how the link structure of an App like this should work. One thing is you certainly want to be able to provide permanent links to the uplaoded photos so your users can share their photos with others. The link should be as short as possible. If your links are 500 characters long it’s probably not going to look that great when you paste them in an email or an IM window for a friend to take a quick look. However, facebook and flickr don’t seem to mind too much about how long the link is. I do have to point out, however, that they probably store billions of photos and my system would certainly not scale for them. With that said in my demo you’ll notice every photo gets a random five character alpha-numeric (case sensitive) link that is directly pointing from the application’s web root. This is the same system imgur uses for their gallery links as well. Except that I noticed a few deficiencies in their method. For one thing they don’t seem to care much about the extension you use to directly view the image. For example, if I upload an image to imgur I might get a link that looks something like http://imgur.com/abcde which would give me a web page with my photo and some information about it (much like you see in my demo) and the direct link to my photo would probably look something like http://imgur.com/abcde.jpg or whatever the file extension was. However, if I were to visit http://imgur.com/abcde.gif or even http://imgur.com/abcde.pngabc I would still be able to see my image. However, I can’t go to http://imgur.com/abcde.exe because that seems to give me an image stating the requested image was not found or has been deleted. Upon some investigation I noticed that their servers return the Content-Type header based on whatever the extension you supplied is as long as it starts with an extension they accept such as jpg/gif/png but it doesn’t matter if it’s proceeded by anything else. This is actually pretty bad, because the file still comes back with the same exact MIME type I uploaded it in. So clearly they aren’t providing the same image in various formats just conforming to some loosely thought-out rewrite rules. Basically I have similar features in my demo where the webserver (in my case Apache) uses rewrite rules and conditions to verify the requested URL and route the request to the proper PHP script and with a little magic you have access to all of your photos from the webroot even though the image files themselves aren’t even stored in the same physical directory as the webroot on my server. So far this demo has only been up for a couple of weeks from the date of this blog post and at around 1,000 uploads and 30,000 views it seems to be reasonably responsive enough that it proves scalable with a little work. I’m using GOCR for the Optical Character Recognition stuff, which is an open source tool developed under the GNU Public license and you can visit their website here to download it or to get more information if you’d like.

A Final Note About Photos

Well, that about covers what I wanted to say about sharing photos on the web and building applications that can do this nicely. Just keep in mind there is a lot to be done with photos that we are yet to uncover. So be prepared to make some good use of these features in building your own applications. I hope my demo gave you some ideas to work with. They are all feasible and not incredibly difficult to implement as you can see this brief demo only took me a few hours of work to put together and works rather well for it’s purposes. Sorry for the horrible interface though that’s one thing I didn’t have time to actually work on. Do let me know what your thoughts are on this subject and if you have any photo applications you’ve built or ideas you’d like to share.