Data Sanitization Suite 2.0

If you haven’t already upgraded your data sanitization suite, it’s definitely time to get started, before it’s too late. Dirty, unclean, unsanitary data is creeping into your application layer; leaving unwanted residue behind. Users with un-bathed data are everywhere and they’re going to stain your clean databases and persistence storage layers!

If you haven’t already realized I’m being cynical, and truly believe there is such a suite, you should definitely keep reading. If you have, you should keep reading anyway. You might actually learn a few things.

There’s no doubt about it. Users can supply your application with data that can break it whether it’s intentional or unintentional. Whether there is a malicious intent, or not. So we certainly don’t want to blindly insert data from the user into our application layer and allow it to inadvertently seep through our code, like in the case of Bobby Tables. However, the problem isn’t that the user’s data is dirty, but that your code has been engineered in such a way that it has developed a data germophobia. This alluding side effect only exasperates your problems in dealing with user-supplied data.

Data Belongs To The User

First, you have to consider that this data isn’t yours to begin with. It’s the user’s data and the user should have control over their own data. However, the code and the application are yours and you, likewise, should retain full control over your code and your application layers. The heart of the problem really lies in the places where those two things tend to meet and the line between where your code ends and the user’s data begins becomes quite blurry. In fact, they may, very well, be indistinguishable at times. So it’s easy to mangle the user’s data and break it just as it can be easy for the user to intentionally — or unintentionally — break your application. In both cases the intent of both the user and the software engineer is unimportant in solving this problem. What really matters is that the solution allows both parties to retain their respective rights in not intruding on the other’s property.

Code Belongs To The Programmer

As a programmer — and especially as a web developer — you’re always taught that user-supplied data is never to be trusted. Sometimes this loosely transpires into “the user is an idiot“, however, that’s just a misnomer. The collective user-base of a web-based application or Software as a Service translates to more valuable knowledge than any two programmers have, combined. This is usually because the engineer works to solve a problem and the user is just looking to get things done without a big hassle. The two have different objectives, but one also has a broader picture than the other.

The notion tends to be that if you soak the user’s data in enough bleach, hose it down with enough Lysol, and dust/vacuum around it regularly, it shouldn’t be a problem. Speaking purely from an analogical point-of-view, of course. However, not all data is created equal, and not all sanitary products are the same. Have you ever accidentally thrown a colored shirt in the wash along with your whites and added bleach? Just as that’s going to ruin all of your whites the same mistake will likely corrupt a bulk of your user’s data (and probably might break your application) as well.

So the real end-goal here is not “this data is a problem“, but more so that “this code and this data don’t seem to get along“. The most obvious solution is to then separate the two and create a clear layer of segregation between them so that one does not interfere with the integrity of the other.

Examples Of Poor Data Sanitization Practices

One common beginner mistake is to think that stripping things from the user’s data might solve the problem that that data imposes on your code. That’s a problem though, because it ultimately means you have removed things the user (for all we know) had every intention of keeping. To me this just means you’ve introduced a new problem (breaking the integrity of the user’s data).

An example of this in PHP is where you want to output user data in your HTML, but you don’t want to allow the user to inject HTML into your output (breaking your application). Thinking that if you just strip all of the (less-than <), (greater than >) characters, from the user’s supplied data, this will keep your code integrity, is a biased view. What about the integrity of the data presented by the user? Why would you assume that the user had the intent of injecting HTML or performing some malicious XSS injection just because their data contained invalid characters that your code cannot accept? Instead, we should find a way to retain the integrity of both our code as well as the user’s data without posing any vulnerability or crippling of the application.

Code

$_POST["input"] = "X < Y && Y > Z";
$output = str_replace(array("<", ">"), "", $_POST["input"]);
echo "<p>The user said: $output</p>";

Output

<p>The user said: X  Y && Y  Z</p>

The above example in PHP is a horrible idea. This is not something you ever want to do. Now the user’s data has lost all its integrity since, in this example, the user simply wanted to present data that states “X < Y && Y > Z“, but your application as rendered their data useless. Think if this were meant to be a post on a public math forum what the impact of your code would be on your user-base.

Lets consider another example…

$_POST["input"] = "
<html>
    ...
</html>
";
$output = strip_tags($_POST["input"]);
echo "
    <h1>User Data</h1>

    <div>$output</div>
";

The output becomes tainted data…

<h1>User Data</h1>

<div>
    ...</div>

Imagine if this were a public forum for web developers and someone were attempting to present some HTML code as a example to some question? Certainly the intent here is to prevent HTML/XSS injection, but that shouldn’t result in breaking the user’s data either. So lets present a real solution to the problem that doesn’t pose yet another problem.

Escaping Vs. Stripping

$_POST["input"] = "X < Y && Y > Z";
$output = htmlspecialchars($_POST["input"]);
echo "<p>The user said: $output</p>";

<p>The user said: X &lt; Y &amp;&amp; Y &gt; Z</p>

This allows us to encode the user’s data to HTML entities that the browser will not confuse for markup. It means the user’s data appears in the browser just as they typed it and there will be no unwanted intrusion of that data into your HTML. Great, now your code doesn’t break and the user’s data retains its integrity. Imagine that, no vulnerability to your application layer and no data corruption! It’s a WIN-WIN situation. The point remains that we do not intrude onto the user’s property and the user does not intrude upon ours. Here both the application layer and the data layer can co-exist in harmony.

What you don’t want to do is escape the user’s data and then store it in it’s escaped form. For example, don’t use htmlspecialcahrs or htmlentities before storing user data in your database. These are meant as transport mechanisms for the document character set, not to be confused with fortifying your SQL against malicious injection. This just means what you have in your database isn’t what the user supplied you with. You would need to take additional measures to unescape the data back to its original encoding in the event you need to perform any actual work on the data, such as searching, transferring, etc…

Instead all you really need to do in order to clearly separate the user data from your application code is make sure you escape it properly (in this case for HTML) before it gets mixed in with your code. The most common approach to avoiding this blurry line, we mentioned earlier, of mixing the two together is to use a templating system. This would be a place where your HTML can live cozily and accept values to be inserted in the template at will, handling the rendering of this view through abstract method. The data you hand it from your modal would than be transported in the proper encoding and the data itself remains unaffected. It’s also, not just, user-data you’re worried about here, but if any of your own application data might break that HTML, as well. So this abstraction is a fitting solution for a common problem.

The SQL Injection Problem

Of course this brings us to the infamous SQL injection attacks that a user can pose on your application by presenting you with data that might break your SQL code if you were to combine that data with your SQL code in the same way you tried to combine the user’s data with your HTML code. The obvious solution has always been to escape that data before allowing your code and the user’s data to mix and mingle together.

$input = mysql_real_escape_string($_POST["userdata"]);

$sql = "INSERT INTO `table` VALUES('$input')";

mysql_query($sql);

It does the job, but the problem of not being able to clearly distinguish your code from the user’s data remains. Here, escaping user data for SQL isn’t as easily solved with the templating system as it is in HTML. Templating SQL code that stores, retrieves, and operates on the very thing your application code depends on, is quite challenging. The method of escaping has been prone to user-space error for many years. Until DBMS developers discovered some better options for abstracting the process in much the same way you attempt to do with an HTML templating system.

Prepared Statements With Parameters

Using parameters in a prepared statement means much the same to your DBMS as an HTML templating system means to your application’s business logic. The purpose of the template is to serve as an abstract idea of what you want rendered in the view. Your application code might want to do various things with the data before it is presented to the user for output. So separating what ends up on the screen, from what’s going on behind the scenes in your code, is important. Equally as important, is the prepared statement that makes it possible to bind parameters to values that can never be confused as code.

$pdo = new pdo("sqlite::memory:");

$sql = "SELECT `username`,`userage` FROM userlist WHERE `userid` = ?";
$stmt = $pdo->prepare($sql);
$stmt->bindParam(array(1, $_POST["uid"], PDO::PARAM_INT));

$stmt->execute();

Here the separation between your code and the user’s supplied data is quite clear to your database. The statement is prepared separate from the data and the parameter is used to bind some value into the statement, then both the SQL code and the data are sent along separate paths. We can’t confuse the user’s input for the SQL code or vice-versa. The same goal you hope to achieve when you want that data placed in your HTML, but don’t want it affect the HTML code and without changing what the user handed you.

Validation vs. Sanitization

This another prevalent assumption that if I only take from the user what I need I can keep my application unaffected and working smoothly. The information a user supplies on the web can be vast. We input everything from our names, addresses, zipcodes, credit card numbers, phone numbers, even to entire documents on the web. There’s a great deal of potential for malicious intent to try and sneak bad data passed our applications to break them. However, there’s also an important need to keep in mind that your application is all about the user. If all your code is doing to make things safer is drive the user more and more annoyed with the process of supplying their input or uploading their information you are only degrading the very people you wrote the code for in the first place.

/* If you're doing this you are causing your users a lot of pain! */

$name = preg_replace("/[^a-z ]+/i", "", $_POST['name']);

Here are some reasons why this is wrong.

Why can’t my name be Robert Jr. or O’reilly?
Or how about a hyphenated name like Lee-ann?
Why would you assume my name can only be presented by the letters A-Z?
Have you never met a Jérôme or an Aimé or a Noël before?
If you know Afrikaans you should know some vowel sounds in Afrikaans are represented by an apostrophe.
Your mangling people’s name! It’s not yours to mangle…
Mostly, YOU’RE PISSING PEOPLE OFF! STOP!

If the data the user has presented you with, is unacceptable for your application then what you should be doing instead is simply validating that the data is acceptable to you and then use that validation result to determine either (A) the data is acceptable and we can proceed, or (B) the data is unacceptable and we must reject it entirely and notify the user to retry according to our requirements. But under no circumstance should you just change the user’s data without notifying them about it and continue on as if what you have is what they gave you.

However, not everything the user submits necessarily requires validation. As developers, we sometimes like to assume that we know everything and that everything must be validated by us before it should be allowed. This is simply not true. We don’t know everything and we certainly shouldn’t have to be the overseeing party of what is or is not allowed as someone’s first and last name. Or characters the user is allowed to publish to a public forum. This stuff does not break our application code.

In retrospect, there are places where validation is a requirement for the application to function properly. For example, we may need to verify that a user has supplied a valid zip code or postal address on in an order form where they are placing an order for us to ship. If the user enters invalid information there the order can not ship and it presents a problem for the intended users of the application software. We also don’t want users entering one or two letters for their password. This makes the account insecure and opens our application up to attacks. So we may want to validate the user has supplied a password of a specific requirement, like say at least 10 to 20 characters (or perhaps just a lower-bound to prevent people from easily guessing a password). We might want to ensure the user includes at least one upper case character or one special character as well to increase password strength and reduce ease of brute force attacks. However, if the user’s data does not meet these requirements we should be rejecting the data entirely and informing the user of the problem so that they may retry. You wouldn’t just change the user’s password to have it meet your needs and simply carry on. That wouldn’t make any sense! How is the user going to know you changed their password? So you also shouldn’t change the user’s data anywhere else unless you’ve made the user completely aware of what you’re doing to their data so that they have an option to decline or at the very least may chose not to supply their data under such conditions.

$regex1 = "/[a-z0-9]{10,20}/i";
$regex2 = "/[A-Z]/";

if (preg_match($regex, $_POST['password']) && preg_match($regex, $_POST['password']))
{
        /* Password is acceptable */
} else {
    /*
        Password is unacceptable
        Reject and inform the user
    */
}

Also, don’t go out of your way to make it very difficult for the user to meet your requirements. For example, have you ever seen an online order form where you’re asked to enter such information as your phone number or credit card number and prompted not to use dashes or spaces? Why should this be a burden the user is faced with? Here you aren’t really validating anything but the user’s ability to follow instructions. Have you ever heard of a regular expression? Have you ever heard of client-side code that improve the user experience? For example, if you want the data sent to your application in a specific format why not make that a part of your front-end? You can use separate input fields to make it clearer to the user. You can also validate the formatting according to regular expressions on the back-end. But don’t constrain the user experience when you have other options that can still ensure your data validated and keep the user happy at the same time.

Do Computer Science Geeks Need Glasses?

You’re a CompSci Major And You Don’t Know It

In my provocative attempt to get the attention of a computer science major student I have had to go to a great lengths to make a subtle point. Unfortunately for me this took up a bit of my time. Fortunately, for the person involved in the discussion I probably got through to them. Although, you can never be too sure. Some people simply refuse to accept that they may be wrong about something out of delusions of grandeur, perhaps self-fixation, or even for reasons of pride. Whatever the case may be it stands to reason that no matter how well you think you’ve learned something you may still be wrong. Medicine is one controversial field where this notion holds true on a constant basis. Physicians and researchers in the medical field find out that what they thought they knew actually isn’t true only many years after their work has been published. The same can be said about the field of computer science.

Allow me to explain how this discussion began. I was speaking with a programmer – let’s call him Foo – on an online forum for PHP. Now, Foo, is new to PHP. He’s trying to optimize his code and has posted some PHP code on the forum looking for advice from more experienced programmers in PHP. So far Foo seems like the typical case where they understand the language syntax and just need assistance making the working code better. Now, another programmer steps in on the forum – let’s call him Baz – and contributes at length with suggestions that seem to me like pathetic micro-optimizations that aren’t really going to help Foo. His suggestions begin with things like use shorter variable names, don’t define keys for arrays, use include and not include_once, and use switch and not if constructs to make your code faster. I start to see Baz is clearly not as experienced at PHP as he’s trying to make himself out to be.

Let’s examine why I think so and why Baz might be very wrong.

Will The Real PHP Please Stand Up

First, Baz is making some non-optimal suggestions that don’t necessarily improve the performance of your PHP code, but might actually break it. For example, suggesting the use of include vs. include_once for performance reasons is not a reason at all. Since both do different things they have different uses. You should not be basing your use of either construct on performance reasons alone. You definitely want to use include_once where your intention is never to re-parse the same file twice; where the order of the calling and called scripts is not clear.

Second, it can not be easily determined that using either include or include_once will hinder performance or improve it. The include_once call itself is merely a wrapper around include that checks the list of already included files. Yes, you’ll be hitting the hashtable, but if you don’t already know it yet PHP hits that hashtable for virtually everything you do in PHP. The hashtable was micro-optimized to be fast for this very reason. Additionally, such a performance hit isn’t even considerable when the need for include_once is apparent. This is like saying to improve my car’s gas mileage I’m going to take out my backseat. Sure, removing any weight from the car means less energy required to move said car, but when you understand the mechanics of the modern motor vehicle and apply just a little bit of common sense you soon realize this is a pretty silly move when you have passengers on board and they are uncomfortable or even injured on the ride because you don’t have a back seat in order to spare $0.01 of gasoline costs on a 15 minute trip. If you’re so concerned with cost you really need to seek out an entirely different solution. Like move away from the horrid design of the combustion engine altogether!

Third, suggestions like a switch is faster than the if construct are completely unfounded and very much baseless over-generalizations. The semantics of the if and switch construct are pretty similar in most high-level languages including PHP. They may differ slightly, but most of it comes down to simple evaluation and jump operations.

When Baz began explaining to me that I’m an idiot that doesn’t know what he’s talking about, I merely shrugged it off as someone who was about to rant on a public forum that there was an obvious performance benefit between switch and if in php. So I moved on. Then when Baz realized I wasn’t responding to his rants he began daring me to disprove him. This is a common false burden of proof fallacy since he is the one posing the argument it really is his burden to prove himself right and not make me do the work to prove him wrong. But, in good nature I laughed and proceeded to prove the poor compsci student wrong. To do so I showed him the branch analysis of an if versus a switch construct using the same 4 expressions to simply check for a true or false and print a single line.

PHP IF Branch Analysis

PHP Switch Branch Analysis

Now, here’s the kicker. If you want to be pedantic about it… The switch will actually cause PHP to generate more opcodes than using the if/elseif construct in this case. This still really says little about performance though. Since they ultimately just lead to a simple branch analysis in the end. All PHP is doing here is evaluating the expression given, to a boolean value of either true or false. If it is true then it executes the opcodes for that branch, if not it moves on to the next branch.

Here’s the code for both scripts.

IF – test1.php

$baz = false; $bar = 0; $foo = ""; $var = true;
if ($baz) {
        echo "We have baz!\n";
}
elseif ($bar) {
        echo "We have bar!\n";
}
elseif ($foo) {
        echo "We have foo!\n";
}
elseif ($var) {
        echo "We have var!\n";
}

Switch – test2.php

$baz = false; $bar = 0; $foo = ""; $var = true;
switch (true) {
        case $baz:
                echo "We have baz!\n";
                break;
        case $bar:
                echo "We have bar!\n";
                break;
        case $foo:
                echo "We have foo!\n";
                break;
        case $var:
                echo "We have var!\n";
                break;
}

So all of a sudden, Baz doesn’t seem to have a leg to stand on and wants to start arguing branch analysis theory with me and vivaciously explaining what little he seemed to remember from his professor in his last ASM class. I could tell he was taking out the book at this point, but rather than be drawn into a pointless debate about things that aren’t helping the status qua, I tried reminding him to get back on topic and help out Foo, instead of trying to convince me he was a smart compsci student. Baz seemed to be infuriated with this and resented getting back on topic. To the point where Baz had now been banned from the discussion and continued to pursue me in private sending me quote after quote from his books.

To indulge the poor fellow I offered some pointers on what it really comes down to when you step back and look at the bigger picture. PHP is not a very efficient way of doing things to begin with. PHP is built on share-nothing architecture. It’s an interpreted language, which means you’re basically recompiling your program from scratch every single time you want to run it. It doesn’t break down to simple x86 ISA in the end and you can’t justify low-level micro-optimizations in PHP since it will ultimately just end up breaking your code in the process of trying to make it faster. The smart PHP developers, the ones who really know PHP inside and out, will always tell you don’t try to outsmart the interpreter, because the interpreter will likely keep outsmarting you.

The Interpreter Is Out Smarting You!

Take people who try to force references into their code as a good example. They think that by using references everywhere they will save on memory and will ultimately make their code faster in ways the interpreter couldn’t. Let us examine the following code snippet to see why this can be a horrible idea and isn’t really offering any benefits worth pursuing.

$array = array(1,2,3,4,5);

foreach ($array as &$value) {
    /* Do some stuff here... */
}

Alright, so this looks great. We didn’t have to use more memory and we can modify the array values in our loop, right? Wonderful! Now we need to just finish this code with one last loop where we’ll print the array elements to the page in a table.

echo "<table><tr>";

foreach ($array as $value) {
    echo "<td>$value</td>";
}

echo "</tr></table>";

The result of our code is now:

1	2	3	4	4

If it isn’t already obvious to you what happened here… You basically just got outsmarted by the interpreter. In trying to outsmart the PHP interpreter you ultimately broke your code and now have an undesired side effect. This is completely expected behavior, by the way. The reason for this is because $value is still a reference. The reference never went away, right? You made it a reference so its use anywhere else in your code still keeps it a reference. And since you decided to use it in the next foreach loop PHP is still assigning the value of each element upon iteration to the $value variable and thus upon the last iteration we’ve now reassigned the value of this variable by reference. Yes, you brokeded it!

Not to worry though. When you’re iterating over an array with foreach PHP is already using the same amount of memory you would use as if you assigned the variables by reference anyway. Why? Because PHP is using copy-on-write. Which means it doesn’t use the memory unless there has been a write operation that has now changed the values of one of the variables. Otherwise the variable is nothing more than an extra refcount for the ZVAL.

Goodbye Baz

Now, Baz eventually gave up and agreed that what he was arguing was not only baseless egocentric banter, but also not very helpful to Foo. I explained to Baz that he should never think that information from a single source amounts to intelligence. It’s just information. Intelligence needs to be gathered from multiple independent sources and corroborated through peer-review. Otherwise, it’s just pointless to say that it’s intelligible when you can’t even get your peers to agree on it.

So the next time you get a compsci student trying to tell you off like he’s the boss, ask him or her an intelligible, objective, question. If they can respond with an intelligent and completely objective answer they might be worth listening to. Otherwise, tell them to go back to school. Computer Science students that have the attitude “I learned it this way and it can never possibly be right any other way” are always going to find it difficult to get work once they step outside of their school and into the real world. Word to the wise: CompSci Geeks DO need glasses!

Renewable Energy: You Are Being Lied To

It seems today that the world has been blindly led to believe that we are dependent on non-renewable energy, because there simply is no other alternative energy source that can power the world. You’ve been lied to! Since the dawn of the 20th century the western world has emerged out of the dark-ages, so it seems. The United States being the leader in delivering energy to power homes, businesses, and entire cities with non-renewable energy sources such as fossil fuels. The problem with fossil fuels is that they are inefficient, produce byproducts hazardous to our environment, because they are based on explosion energy (combustion). You may have been led to believe that the world is just now emerging out of the dark ages for the first time. That man has seen only primitive civilizations, without advanced technology, without advanced sciences. You’ve most certainly been lied to and you probably never bothered to question it!

The Truth

The truth is the world has enough renewable energy sources to power our planet and light it up like a Christmas tree for many — many — centuries to come. The truth is renewable, clean, and natural energy is far more efficient than any of the fossil fuel based energy we use today. In fact, the truth is we are not the technologically advanced civilization we think ourselves to be. We’ve been taught in school, for many decades now, that ancient civilizations were primitive people, using stone and copper tools, hunting and gathering, and eventually we built up to civilizations that cultivated the land and so began the manufacturing age. As though mankind has been on a linear course of progression for thousands of years and it’s just now that we have discovered science and technology and things ancient civilizations could not possibly have fathomed. As if we are thinking up computers and massive global networks for the very first time in mankind’s history. Evolutionists would have you believe this fits perfectly with their theories and politicians lining their pockets with the fat-checks of money-hungry oil conglomeration can do little more than back such ridiculous notions. Not based on any shared beliefs or ideologies with evolutionists, but strictly based on math.

If the world were to discover that oil and coal are not the greatest source of energy there would be wide-spread downfall of international energy corporations and all of their infrastructure. Many interested parties would take on insufferable losses and their business would surely vanish like the ancient civilizations we no longer see today. I imagine, that if the ancients were still around today they might actually be the ones laughing at us and calling us the primitive ones. Why is it so inconceivable that other ancient civilizations might have discovered the immense benefits and substantial gains from building global long-term infrastructures that would harness and support technological advancements? Is it really so inconceivable that ancient civilizations could have had sciences and technologies just as advanced as ours, if not more? Unfortunately, science can not teach us what could be, just what is. Yet, it is through science that I can show you just how much you have been deceived.

The Advent of Electricity

In order to see where we went wrong with energy, we must first begin our trip more than 120 years ago when electricity was just taking form in the heart of the United States. During the late 1800′s America was just beginning to form infrastructure that was quickly changing the world. The rapid development of the telephone and now electric power plants. This was at a time when Edison was just getting started.

However, most of us only know one half of this story. We tend to have a biased opinion about energy problems today, primarily because we have insufficient information to actually make an informed decision about the subject at hand. We are ill-equipped to formulate opinions, because we lack the analysis of the full story.

In the late 19th century a man named Nikola Tesla began bumping heads with Thomas Edison. Tesla, worked for Edison at the time. Edison wanted to use DC – Direct Current – which was inefficient and proved difficult to supply due to its limitations in transport. Tesla wanted to use AC – Alternating Current – which proved superior to Edison’s Direct Current. A direct current travels in one direction whereas an alternating current can travel in many directions at the same time. However, Tesla was hardly recognized for his inventions and radical innovations. He was considered outcast from the science community and ultimately went into seclusion. It was not until decades after his death that people began to truly realize the genius that was Nikola Tesla.

Tesla, none-the-less, was a visionary. A mind well beyond his time. He tried demonstrating his device (the Tesla Coil) for using Alternating Current at the World Fair in 1893 — World’s Columbian Exposition in Chicago.

Within the room was suspended two hard-rubber plates covered with tin foil. These were about fifteen feet apart, and served as terminals of the wires leading from the transformers. When the current was turned on, the lamps or tubes, which had no wires connected to them, but lay on a table between the suspended plates, or which might be held in the hand in almost any part of the room, were made luminous. These were the same experiments and the same apparatus shown by Tesla in London about two years previous, “where they produced so much wonder and astonishment”.

Barrett, John Patrick (1894). Electricity at the Columbian Exposition; Including an Account of the Exhibits in the Electricity Building, the Power Plant in Machinery Hall. pp. 268 – 269. Retrieved 29 November 2010.

What a remarkable idea, right? Wireless electricity developed in the 19th century? We’ve only just began to experience wireless technology in mainstream society not more than at the turn of the 20th century. Yet, here is a man far removed from our times that already thought of this idea. In fact, Tesla built a tower, called the Tesla Tower (also referred to as Wardenclyffe Tower) in Shoreham, Long Island, New York for wireless telecommunications that operated between 1901-1917. Eventually, his investors, J.P. Morgan, cut off his funding because Tesla wanted to use this tower for allowing people to transmit telephony and broadcast wirelessly and for free. Of course his investors could not make a profit this way so they pulled out and the tower was never complete. Imagine, wireless, limitless, global communication, that was free! Does the Internet start to sound like less and less of a new revolutionary idea, now? Perhaps you might still not think so, but what will surprise you next is that this wasn’t even the first time someone has proposed, or even built such a technology!

Back To Ancient Egypt

In order to understand even further, where we went wrong with today’s so-called “energy crisis” we must travel even farther back in time to the time of Ancient Egypt. The time of the Pharaohs. A time when civilization had clearly left its mark on the face of the world. Man-made structures. Tall, magnificent structures left behind by an ancient civilization that is no longer around today. Heritage, culture, language, artifacts, and even a tremendous wealth that were all left behind for us to discover. From these remnants what scientific evidence have we actually discovered about this civilization?

You may have been led to believe that the Pyramids of Giza were used as tombs for Kings. If this is what you were told then you were lied to! Albeit, it’s quite possible that the people who lied to you were themselves lied to or just merely misled. It wasn’t an intentional sinister lie. It was one of misunderstanding and ignorance. Archaeologists have been hard pressed to explain the many mysterious of the Pyramids of Giza, because they were among the largest and most mysterious of all the Pyramids we have discovered today. Despite the mostly mythical notions about these Pyramids in Giza, Egypt being used for the tombs of Kings, the truth is they were the power plants of the ancient world. Yes, Pyramids, were power plants of Ancient Egypt. How? Science tells us the story we were too stupid to understand when we first started exploring them.

These are hieroglyphics from a temple in Egypt depicting that Pharaohs used light-bulbs long before we ever thought electricity had been discovered. Furthering the evidence for this claim is that if the ancients did not use electric light bulbs (depicted by the Pharaoh in this hieroglyph holding a bulb connected to a wire which is powering the luminous bulb from the Sun God Ra) they would have had to use flaming torches to see inside the corridors of the Pyramids. Torches would have left a residue of soot inside the chambers and corridors of the Pyramids. However, there is no evidence of any soot. The ancients did not see in the darkest bowels of these structures by way of fire as we once had believed.

It seems that the ancient civilization we deemed inferior to our own in technology and scientific advancements were so superior that ironically we failed to realize by just how much. Not only is there scientific evidence of technology apparent in the inscriptions left-behind from this civilization, but there is proof in every aspect of their work that we have only recently begun to unfold within the last few decades. For one thing, the mummies discovered in Egypt were all found in burial places such as the Valley of the Kings, which is a site far far removed from the Pyramids of Giza, or in other smaller pyramids and tombs that are not the great pyramids of Giza. There was not a single mummy found in these Pyramids. In fact, there isn’t even a single Hieroglyphic marking inside the Pyramids. Most tombs, temples, and burial places exhibited explicit inscriptions about their purposes; clearly marking that they were the resting places of Kings or Pharaohs. The Pyramids of Giza, however, did not. They remain a deep mystery – to this very day. Some of their secrets are yet to be discovered.

Think of it this way. If you were building a nuclear reactor, would you decorate it with art work and describe its uses and purposes openly? Surely, you would not. Today’s nuclear reactors and power plants are tightly secured structures; their secrets close-guarded by those only granted clearances into its deepest borders. Like-wise the Pyramids were only unraveled through close observation and careful study. Early Archaeologists only depicted these structures as the tombs of Kings because when they were questioned about their purposes and significance to the civilization they could come up with no other logical explanation. So they simply tried to deduce what might have happened to the mummies by hypothesizing that the sarcophagi and/or their mummies had been stolen by grave robbers along with any riches they may have found within. However, there is no evidence of there having been any riches in these structures. Would you store money in a power plant? There is no evidence of any sarcophagus lingering anywhere about in the corridors or chambers inside of the Pyramids. What we found instead in the Great Pyramid of Giza was what looked like it might have been a place for a sarcophagus, but turns out to have had a much different purpose.

Buried deep beneath the Great Pyramid of Giza sits a chamber with a single granite compartment showing clear evidence that it had been carved from a single slab of a special kind of granite. Granite is naturally radioactive. Certain types of granite have proven to have up to 10 PPM (parts per million) uranium. The zig-zag style corridors inside of the pyramids indicate passage way to aquifers leading from the Nile river beds to the underground chamber beneath the Pyramids vast structure. The water was used as a conductor in the production of electricity. Though the Pharaohs were not manufacturing electricity the way we are used to today. They were harnessing natural electricity. In Physics, we refer to this as subtle energy. It’s the use of implosion energy, not today’s more common explosion energy that set this civilization apart from ours by centuries and centuries of being in the dark ages. The Pyramids were coated with limestone from the outside. Limestone is a poor conductor of electricity. It was used as an insulator (in its pure calcium form). The granite, being a good conductor of electricity was used throughout. The top of the Pyramid was fitted with a huge block of gold as a cap. This is no longer present today, but at the time before it had been removed – or stolen – it was believed to have been used to conduct electromagnetic energy from the sun (the ionosphere). Gold is a perfect conductor of electricity and this would have been the equivalent of today’s solar-power cells used in so-called green-energy. The Pyramids, however, did not just produce or harness the suns energy or the planets natural energy – they had an even greater functional purpose. They were engineering feats of the ancients of our world to build infrastructure that would supply all of Egypt with wireless, safe, and limitless energy. They were crafted with the utmost precision.

Imagine, not only could you use these structures to power light bulbs and other tools, but you could power the people. The human body is the perfect conductor of electricity and can be used to safely pass electric energy current through the skin without harming the human being. Portable, efficient, safe, and best of all renewable energy! Is this not the mark of a genius civilization? A technologically advanced civilization? Yet, today we pride ourselves in having built microchips that produce immense heat, cost untold amounts of electricity – a result of harmful fossil fuels, and plead to be advanced.

The Pyramids are now theorized to be the greatest proof that ancient civilizations were indeed in position of advanced technology that even today we have not managed to acquire. In fact, with all of our technology today we still can not reproduce these magnificent structures that lay and remain in their place for more than 5,000 years!

The Great Pyramid of Giza is made up of more than 2.3 Million stones. Each stone weighs in at around 2 or 3 metric tons. That means this structure required people to cut, move, and assemble more than 6 or 7 Million tons of stone. By comparison, each of the World Trade Center Towers of New York City weighed around 500,000 tons. That’s an order of magnitude less than the Great Pyramid of Giza. Additionally, we know where the stone came from because of its mineral compositions having been tested. They were hauled from mountainous regions hundreds of kilometers to the south of Giza. Some Archaeologists might have you believe that these stones were moved using ropes, and piled on top of logs and raised with simple pulleys. Dragging two metric tons across the desert over a hundred miles? Sure, these men and woman all had Ph.Ds from fancy Universities! Lets give them credit for their imagination if nothing else, but in what science can you convince the world this explanation can be scientifically proven? Where did we find millions of logs of timber in the middle of the dessert? Further more, how many man hours do you calculate it would take hard laborers to move all these stones? It would have taken centuries at their populations. Beyond that there is evidence of pure craftsmanship everywhere in the Pyramids. The stones are cut to a fine edge with what could not have possibly been accomplished using mere stone and copper tools of simpletons. The stones are assembled with such precision that it defies all plausible rate of error, beyond that of which our modern-day engineers can even fathom. This is simply outstanding evidence that we haven’t even bothered to consider the possibility that perhaps it is us who are the inferior civilization. Perhaps we are only assuming the ancients were living in the dark ages, because their knowledge had been lost and it is our ancestors that were the ones thrown into the dark ages only of late. Now that we are merely just emerging from these dark ages it can not be so difficult to believe that previous ancient civilizations may themselves have once been at the height of technology and scientific advancement. Perhaps even so much so that they surpassed anything we have done to this very day by centuries.

You Are Not Dependent On Fossil Fuels

The lesson to learn from all of this is that fossil fuels are not the only source of suitable energy that can power our planet. Not only did scientists find proof of this technology among the Ancient Egyptians, but even the Mayans, the Chinese, and ruins discovered in European countries demonstrate the existence of Pyramid structures all around the world that may very well have been used for the same purposes. Tools that would compare to today’s only recently sought technological advancements like the quartz crystal, power tools that can cut through hardened materials like granite, and much much more.

However, as long as there are greedy people in the world who would have an interest in controlling the worlds energy markets and controlling you to continue buying from them what seems like a product that can be depleted and run out – there will be such gimmicks to try and keep you in the dark about how much we can do without our horrible idea of energy today. The harm and the costs that it has burdened us with is beyond unacceptable.

It’s no new notion that we should turn to alternative, clean, renewable energy sources. We’ve been arguing and debating about this for decades, perhaps even centuries. Take a look at what the smart corporations are doing today about the energy problem. Google has invested a great deal of money in finding clean, renewable energy sources. They have even recently started a subsidiary – Google Energy – to fund this initiative and perhaps even pave their way into reducing the worlds energy costs through clean renewable energy. Google actually finds locations for its data centers in Asian countries were a steady flowing water supply, such as a river, or an ocean is nearby. It engineers and powers its resources based on ancient technology (building damns for power). Natural energy is all-around us. It flows freely throughout the earth in ways we are only beginning to discover. Tesla discovered this in the late 19th and early 20th century when he demonstrated how his device could power a light bulb without wires by allowing the alternating current of thousands of volts to flow through a human body and into the bulb, without harm to the person. Today, trying to touch a high-voltage wire of just a few thousand volts would surely kill you. Tesla’s Tower was powered by aquifers very similar to the ones the Ancient Egyptians used in their Pyramid. Today an unfortunate natural disaster in Japan nearly caused a nuclear melt-down and put millions of innocent people in harms way. There is no need to even mention the war, death, and destruction crude oil has caused us in the last few decades alone. Why are we bringing harm upon this world in order to help its people? If energy really is supposed to help us produce technology that helps and not harms the civilizations of this world then why doesn’t it?

I urge all of you to ask more questions and accept less things at face value. I encourage you to explore the true nature of science and technology as you are being taught in your schools, your homes, your places of business. Where does the world’s interest truly lie and who is trying to simply cover your eyes in order to fill their own pockets? If we were not really taught the truth in history class — how wrong could our science classes have been?

The Internet Blackout: SOPA

Today a large number of popular websites have participated in the Internet Blackout used as a protest against SOPA. It seems that while some websites have decided to block their service entirely from the public in order to raise awareness about the SOPA act and in an urgent call to get Americans to act, others have taken a more subtle approach.

For example, websites like reddit, mozilla, imgur, and wikipedia‘s English site have chosen to completely blackout from the public. While wikipedia is easier to get around than others sites like reddit, mozilla, and imgur are pretty much going to leave you empty handed for the day. Google simply blacked out their logo today and provide a direct link on their front page. If you’re visiting Google for the first time today you might have been redirected to the announcement page to contact your local senator. Beyond that they don’t seem to nag you like other sites are doing. Sites like WordPress, php.net, and Google took an elegant, but effective approach to the blackout. They demonstrated their anti-SOPA while still keeping their visitors happy. Sites like Bing and Yahoo, however, don’t seem to have participated in today’s blackout in any noticeable way.

Internet Blackout - SOPA

Why You Need a Database

There are a lot of developers that start off building their applications with the notion that a database is only necessary if they have a lot of data to work with or that the data they have will be easier to manage if they can avoid the complexities of building and maintaining a database or dealing with a DBMS (Database Management System). In the area of web-based development, this is rarely the case. The reason for this is that web-based applications tend to grow very rapidly. This is easy, because there are billions of people with access to the Internet and virtually anyone with access to the Internet usually gains such access from a web-enabled device. Having access to the Internet has become synonymous with having access to the world-wide web. Since the number of potential users is so huge the potential for data is equally huge. Not only that, but beyond the sheer amount of data that maybe collected from users of the application software and stored for use by the system there is the factor of maintainability. Databases make organizing and maintaining long-term data easier. This comes in several forms. Without a database solution you have to worry about concurrency issues for replication. You would also have to consider race conditions, access time, permissions, and scalability among others.

Databases Are Overkill

For those who start off building small web-based applications or even trying to put together a tiny CMS (Content Management Systems) they sometimes fall victim to the illusion that having a very small amount data would mean that building a database for this data would be overkill. This is simply not true anymore. Today databases are easier than ever to build, grow, and manage. With lite-weight solutions like SQLite you actually improve on performance with small amounts of data and make it easier to manage. SQLite is actually a small foot-print library written in C that implements an embedded DBMS. It’s only a few hundred KB in size and implements most of the SQL standard. You can use it to store databases in memory or on disk and still get the full benefits that relational databases offer with a minimalistic foot-print and without compromising on performance for small data sets. It’s adopted by PHP, Python, Perl, Ruby and even Javascript as well as many other languages. So there really is no excuse to avoid using a database when the solution is widely available in so many popular platforms and especially in web development.

Databases Are Slow

This could not be farther from the truth. A relational database can maintain indexing for records across different tables. This means rather than looking through the entirety of the data set and then trying to expose some underlying structure in order to find a particular set of data the relational database takes advantage of composing structures as you build your data sets. These structures make things like fetching a record with a primary key much much faster than you would get by using a flat-file solution.

Lets examine the alternatives. Even if you had a very small amount of data – say just a few hundred lines of text. Even if the data structure was overly simplistic – we’ll assume each line represents what would be a single row in a database table. Even if the data will only ever be maintained by a single developer – you. You are still overlooking so many problems that are not easily solved by using a flat-file to maintain this data. First lets consider the race-condition. You have a script that opens a specific file on the server and appends a new line each time a record is added. The script can also open the file for reading and retrieve the entire contents of the file into memory. The script can then do any necessary sorting and filtering to return the required data sets to the user. The most apparent problem with this approach is the race condition. It is entirely plausible that two requests could be made simultaneously to the same script – one to open the file for writing and append a record and one to open the file for reading and retrieve the data. If the data is stored into memory before the line is appended the result is stale and potentially corrupt. If the new data is appended before the read, no problem. However, what happens when you want to delete a record. Now the problem is three-fold. If three individual requests all come in at the same time – one to read, one to write, and one to delete a record – it is now likely the case that your entire data structure has been corrupted. Remember that HTTP is built on a request-response model and no two requests are treated as if they are tied to any previous requests. So there’s no central point of control over your script’s ability to manage which process can access the data and to what extent.

In a DBMS, on the other hand, the control is transferred away from the script and to the central management system of the database. The DBMS then gets to decide how requests will be served and the order of treating the data. This creates more dependable data that has a far lesser chance of corruptibility. Now, it’s entirely possible that you may not be concerned the integrity of your data for a small application, but then you might as well not waste your time building it.

I’ll Use A File Now And Learn To Use A Database Later

If you’ve said this phrase it’s already too late. It doesn’t take a lot of time to get started with a database in the first place. If you’re using languages like PHP, Python, Perl, or Ruby you probably already have the necessary libraries installed on your system to work with a database. These libraries and drives are usually packaged with these software stacks as standard. It’s actually uncommon to not have some DBMS solution already available in most of these environments. So why would go out of your way to reinvent the wheel when the solution is already at your fingertips? Not only that, but it takes very little time to set these DBMS solutions up and get them to run smoothly on virtually any platform. You will probably spend more time trying to write a script that stores, retrieves, sorts, filters, locks, and validates data using a flat file then you would installing the DBMS and getting a simple schema started.

If you’re using PHP interfacing with a database has become easier than ever. It only requires a couple of lines of code to open a database connection to virtually any database you have a PDO driver installed and loaded for in PHP. So whether you’re using SQLite, MySQL, PgSQL, etc… you shouldn’t need to spend a lot of time learning how to interface with each of these databases if you simply stick with the PDO extension. You use the same functions regardless of the database. This is opposed to having to learn the individual database-specific extensions in PHP to interface with each of those databases. Not to mention PDO supports many of the popular new database features such as prepared statements and is a lot easier to learn and use than extensions like MySQLi.

PHP and Databases

Being a PHP developer, I also take notice of many PHP developers that tend to have the misconception that when they start using a database (usually it’s the case that their first database is MySQL) they should start by learning the old mysql extension in PHP. This is simply not true. Some of the underlying reasons this is widespread, however, is mainly due to the fact that the old mysql extension has been around for quite a long time in PHP and it’s fairly common to see a lot of PHP code demonstrating the use of a database in PHP using this extension. It’s also become familiar to a lot of old PHP developers and is bound to be present in their older applications. However, the use of the old mysql extension is highly discouraged for new development. It’s an old extension that’s no longer well-maintained and has been planned for deprecation for years. There’s no guarantee that if a new bug creeps up that someone will go back and fix it. This leaves your application vulnerable and exposed. If the code base gets large enough this might leave developers scrambling for migration path. Additionally, the extension does not support prepared statements or parameterized queries. This makes things like making sure you properly escape user data to avoid SQL injection, prone to error. The extension lacks in many areas that are not conducive to future development. Learning the old mysql extension first before you learn the new improved mysql extension in PHP or before you learn PDO will gain you nothing. In fact, it will cause you to have to unlearn some of the very poor design of the old extension and its implementation details in order to become more accustomed to the newer extensions.

Some developers also complain that PDO seems too complicated or more difficult to use than the old mysql extension. This might come from the lack of understanding as to what PDO even is or how its used. Since PDO can only be used with the newer OOP features of PHP (you have to use objects and methods instead of procedural-style functions) it makes it seem unapproachable or even scary to developers who aren’t used to OOP in PHP. There is also the idea that PDO has a lot more features due to its vendor-agnosticism and the fact that requires further configurations such as installing and loading the individual drivers needed for interfacing with your specific database (where the drivers aren’t already packaged or loaded). I can understand the intimidation, but most of this has been alleviated with new versions of PHP coming pre-packaged and loaded with most of the popular drivers and the documentation offering up examples that are now easier to follow and get started with. Most of the intimidation is actually coming from having to unlearn old habits that older extensions like the old mysql extension once taught.

Once you get past the initial intimidation phase and actually get started with a PDO and with a database you’ll find that it doesn’t take nearly as much time as you’d think to get up and running. Most of the reservations people have are 90% of what’s holding them back. Not that the investment to get started is actually that significant. Beyond that you’ll find that learning to build on data normalization not only make development easier, but makes your users happier. When you can organize and maintain data that’s more clearly structured and accessible you can serve your users more effectively and efficiently. That will keep users coming back and eventually help you grow your application!

S	M	T	W	T	F	S
« Apr
		1	2	3	4	5
6	7	8	9	10	11	12
13	14	15	16	17	18	19
20	21	22	23	24	25	26
27	28	29	30	31

Sherif's Tech Blog

Just another guy on the Internet with a keyboard…