New Web Order - Nik Cubrilovic

Last week Mike and I had a quick but interesting chat on the phone with Joe Hewitt of Facebook about iPhone development, the new Facebook iPhone interface and Joe’s iUI iPhone work. Joe previously worked at Netscape and now finds himself at Facebook after Parakey, the company he founded with Blake Ross, was recently acquired by the cool social networking company. Joe also wrote the indispensable Firebug, a web development debugger for Firefox.

When the iPhone was released, a community of developers formed very quickly and began hacking away and posting code, solutions and tips in various forums online. The most popular and active of these groups was the iPhoneWebDev group on Google. One of the most active participants in the group was Joe Hewitt, and his work along with the work of many others came together as the iUI ‘framework’. It seems that in the first few days and weeks that many of the participants didn’t get much sleep, as new hacks and tips were being posted at a very rapid rate. Joe was releasing updates to iUI almost every few hours, and the code that he released has gone on to form the foundations of many iPhone applications since released.

Download (26:32 min, 6.07 MB)

I just posted on TC about the Facebook code leak. PHP has always been notorious for sometimes not processing requests poorly and sending back the source code for pages to the client. Because of the way mod_php works with apache, if mod_php fails in intercepting and processing the request, then apache will just serve it back to the client as an ordinary text file. I could go into the details of how this all works and why it sometimes breaks, as well as the causes, but instead lets touch on a few solutions to preventing PHP code from leaking:

Use mod_security to filter output and prevent leakage

mod_security is so damn good that it should be included in apache by default (and there should be some default rules in the default conf files). You can write mod_security rules that will detect if the output is PHP source code, and then prevent it from hitting the wire, instead giving the user an error page. You can also detect other information leakage, and prevent it from escaping. When writing a rule to detect if there is PHP in the output, you can do a regexp against the PHP header tags (eg. ‘< ?php' and '?>‘) or include a special token in your PHP that identifies it as source code (eg. have the comment /* THIS_IS_PHP_SOURCE */ at the top of each PHP page, and if mod_security sees that in the output, kill the response). Here is a simple sample mod_security rule that will filter output:


SecFilterOutput On
SecFilterSelective OUTPUT "<?php" log,deny


For more on mod_security (essential!), see this onlamp article (old but a good intro)

Code should live outside of the web root

You should keep all logic and sensitive code outside of the web root. You can then include the logic files using the include() function. You should already be doing this with any files that store database information or passwords, but you could take this to an extreme and have only a single index.php inside your webroot, which will include a fileoutside of the webroot where everything actually happens, eg:


index.php:

<?php
include('../realroot/index.php');
?>


Change the default file type

By default, Apache will treat files as text/plain - meaning that if the extension of a file doesn’t match a handler (eg. .php files processed by mod_php), then it will send it back as plain text. If you accidently change the extension of a file type, or if an attacker somehow forces an alternate extension, they can retrieve the plain text content. To prevent this, with PHP apps you may want all files to be treated as PHP (and then have certain types handled as plain text). Modify the following directive in http.conf:


httpd.conf:

DefaultType application/x-httpd-php


Deny all outside of the webroot

Assuming your webroot is ‘www’, you want every other directory and file to note be served. Common sense:

http.conf: (or .htaccess)

<directory />
Order Deny,Allow
Deny from all
Options None
AllowOverride None

<directory /www>
Order Allow,Deny
Allow from all
</directory>

It looks like American pressure has finally worked and that Russian music download site AllOfMP3 has closed its doors. AllOfMP3 started as a small service and ran for years without hindrance, but more recently it has become a talking point in the mainstream press as record labels attempted to shut the service down by applying pressure to the Russian government via the US government. The lack of modern copyright law and enforcement has become a high-level talking point in discussions relating to Russia’s entrance into the WTO, with AllOfMP3 serving as an example of what the US government sees as wild-west policies in Russia. The Russians held their ground for a long time, but it appears with Putin about to visit Washington that they have finally swept AllOfMp3 under the rug.

But AllOfMp3 is not lost forever. They have setup at least half-a-dozen other sites all utilizing the same music catalog, accounting system and website, albeit with a different design in each case. MP3sparks is an example of one of these new sites, and those familiar with AllOfMp3 will recognize the layout. Further, if you had an AllOfMp3 account you can login to MP3sparks using the same credentials. You will even find your credit balance from AllOfMp3 in place on the new site. Also, the new sites are accepting credit cards again. I just refilled using my US-based VISA card.

This is a good example of just how futile these efforts are. There was no way that the Russian operators were going to easily let go of what would have been a very profitable operation, and it appears they are in the clear for now with the new sites. I can only imagine how the conversation between the Russian government and the operators of AllOfMp3 went:

gov.ru> Boris, we have a problem, it’s the Americans
boris> what is it this time
gov.ru> they are telling us that we must shut you down
boris> dammit, I am running a legitimate business!
gov.ru> lol!
boris> hehehe
gov.ru> anyway, its the same issue, you aren’t charging your users enough and they want higher royalties
boris> but this is Russia, the average person here makes $300 a month, we can’t charge them 99c a song
gov.ru> I tried telling them that, but apparently lots of Americans are using your site
boris> thats just the free market at work, just like US companies take advantage of cheaper labor in other countries..
gov.ru> whatever, fact is that they are mega-pissed and they keep calling
boris> so, just don’t answer the phone and lets hope they forget about it
gov.ru> tried that!
boris: well, I am sure we could just take the site down but set it up at a different address
gov.ru> that sounds fine, lets just do that
[10 second pause]
boris> ok, done - the new site is at www.mp3sparks.com
gov.ru> great, that should keep them happy until we get into the WTO
boris> well, anyway, I must get back to ripping CD’s. by the way, we refilled your account
gov.ru> hey thanks!
boris> hurrm, I have a funny feeling that this call is being transcribed
vladimir> ahh, you are just paranoid!

Update: It seems that MP3Sparks is down already - probably because of traffic (the site was so busy that the payment processing was timing out). It turns out that AllTunes (www.alltunes.com) is still live and kicking. I find it amazing just how many people are using these sites, and just how well known they are now

Pownce was launched this week to much fan-fare and I signed up to the service and spent all of a few minutes there to see what all the fuss was about (although most of the fuss was about Kevin Rose, rather than the product). In short, Pownce is Twitter with a minor twist - instead of just generic messages you can send four different types of messages. The first is just a normal message, as per Twitter. Second type is the ability to send links, which is just a messsage but with a field for a link URL. The third type is that you can send an event and the fourth is that you can upload a file.

I believe that there are two key things that Twitter has done well that Pownce has overlooked completely. The first is the mobile interface, which was the main reason why Twitter became a success. When you are sitting at a computer, you have a myriad of ways of communicating with others - Twitter made it very easy to communicate with your online social network while you weren’t at your computer. The four features that Pownce offers are all done in a much better way with other services.

The second thing that Pownce is missing is an API. I have written previously about Twitter as a platform, and I think that the importance of Twitter having an API since when they launched has been understated (for instance, Mike neglected to mention the API in his post comparing the two). The Twitter API enabled some of the coolest and most discussed parts of the Twitter universe - applications such as Twittervision and many more.

The API also resulted in third parties producing some excellent desktop clients for Twitter, it meant that researches could study traffic patterns on Twitter and that people like Dave Winer could extend it’s functionality into other areas such as voice. Twitter has a fully functioning and active ecosystem around the main product. Without the API, Twitter would not be what it is today.

In contrast, Pownce has release its own desktop client (which doesn’t work very well) and is a closed environment where not much else is going on outside of the main website application. New features and functionality on Pownce will have to come from the company - and the product will only ever enable usage that fits within their own vision of what the product should be. Pownce does not even have support for RSS feeds, so I must go back to their website if I wish to check my updates (although they are kind enough to send me dozens of emails each day with nothing more than a link back to new messages).

Twitter v Pownce is a great example of why open and easy to understand APIs are so important. I strongly believe that with most web applications, the API should be designed and implemented first - and any interfaces that you build yourself should be an implementation of your own API. That same API should then be available for everybody else to access, and as the service provider you should provide an environment that encourages third party developers as it will benefit both you and your users greatly.

(Note: it is likely that at some point Pownce will enable RSS feeds and an API, but they have lost the opportunity to make the most of their early buzz)

UPDATE: Some people have already asked the same question about Pownce and API support. See Marc Canter’s post on the matter with some interesting comments from Dave, Mike A and Fred Wilson.

UPDATE2: I have 6 invites to Pownce. If you would like one, leave a comment

This weekend in San Francisco Adobe will be hosting iPhoneDevCamp, a Barcamp-related unconference about application development on the newly-released Apple iPhone. I built an Omnidrive iPhone application yesterday and what I found was that there existed a big knowledge gap in terms of what the standard development practices for the iPhone should be, specifically when it comes to interfaces. iPhoneDevCamp seems to be a good opportunity to meet with other developers and to share some tips and tricks. One funny thing about the event is that Adobe are requiring that all attendees commit to a non-disclosure agreement. From the Barcamp site:

Please be advised that all visitors to Adobe are asked to sign the following, on site, either electronically or on paper. This agreement is in place to protect attendees in case they are exposed to any confidential information from Adobe. We do not anticipate that attendees will be exposed to Adobe Confidential information, but the NDA is required for admittance, and only applies to Adobe Proprietary Information.

Almost all of the information at iPhoneDevCamp is not covered by this NDA. Confidential or Proprietary Information (which is covered by the NDA) is easy to distinguish. In all cases, an Adobe employee or representative will explicitly label such information as ‘confidential’ or ‘unannounced’. This notice will take written, or more commonly, verbal form.

It then goes on to outline the terms of the NDA. The NDA for entering the Adobe building seems a bit aggressive, especially as there isn’t a strict definition as to what is confidential - it could apply to something you already know, or something you have already written about Adobe. The NDA on the site states that Adobe can give a simple verbal warning that something is proprietary - it could be as simple as somebody saying that some information is ‘off the record’.

It could well be that Adobe are just really covering their bases, but regardless I think this unconference could have easily been held somewhere where there isn’t a grey cloud of confidentiality hanging over our heads - especially as it is making many of the potential attendees uneasy. Adobe could also do what most law firms and many other companies do, and that is to have a segmented work area in the building, and then a general public area where meetings and events can take place without the fear of anybody accidentally finding out something they shouldn’t have (even though in the vast majority of cases where information does get it - it is through internal employees).

Looks like the event will be a lot of fun, despite a few people on the wiki being concerned about the NDA situation.

Lots of people who haven’t showered in a while hanging out the front of the Apple store here. Lots of built up excitement as the doors are about to open.

6.00pm: Ok the doors are open. Around 200 people lining up, another 500 looking on

6.03pm: Ok, the first guy in line just bought an iPhone! Looks like he went for the 8GB model

6.04pm: correction. that last guy bought *two* iPhones.

6.04:10pm: we can confirm that he bought the 8GB model

6.06pm: ok now the second person is buying an iPhone - also picking up 2 of them. 8GB models again

6.09pm: the third person in the queue also bought 2 iPhones

6.13pm: ok a couple more people just bought iPhones - all very exciting

Read the rest of this entry »

This week at Omnidrive we have been discussing the impending launch of the new Apple iPhone, and what our response as a company should be in terms of support. As some developers know, targeting mobile platforms at the moment is difficult for two reasons. The first is that the development environments are so fragmented between different operating systems (and even within different version of the same operating system). The second reason is that some carriers have a stranglehold on what can and can’t be installed on ‘their’ devices. Verizon is no different to what AOL was 10 years ago, and some of their practices (such as disabling external memory slots, mp3 players or dial-up networking by hacking the devices firmware) are very controlling and anti-competitive. Application development on the web and on desktops is now easier than ever, as operating systems have become almost redundant with the advent of new platforms that provide open abstract layers such as bytecode.

There are many reasons why developers and users prefer open systems and platforms over tightly-controlled proprietary environments, and why open API’s, platforms and protocols usually prevail over closed and proprietary alternatives. This is especially true in modern computing history, as can be seen in how IBM and Sun have re-invented themselves and in the success of open source (and companies that have been built on open source software). When I see the iPhone and how it has positioned itself in the market, I see a position and attitude that is decades old coming back to haunt us. In the iPhone, Apple have provided consumers a mobile device that is as close to a desktop computer in capacity and capability as we have ever seen. The processing power and storage capacity is equivalent to what we had on our desktops not too long ago, and the phone runs an almost full version of a very powerful operating system. The key disappointment is that only Apple, and Apple alone, are able to take full advantage of the capabilities of the iPhone - as they have locked all third party developers out of everything bar the browser. While having OS X on the device is very excited, what difference does that make to a developer (or even a user) if we don’t have full access to the system? Apple’s definition of an iPhone API is letting their users visit other websites, albeit in their browser, which isn’t extensible and provides no support for even Java or Flash.

For Apple it means that they have been able to easily port iTunes, Safari and the Dashboard with it’s widgets - but they are locking it up so that only they can take advantage of the full capability of the device. For developers and users we can publish our own widgets, or users can go to our websites and run our web applications - nothing more than that. This is not too dissimilar to the console market, and even there the situation has improved especially on thee XBOX 360. No other company would be able to get away with releasing such a closed system in a market that is dominated by somewhat accessible platforms - Apple have not only gotten away with it, but have been praised in the process.

There have been a number of application providers who have pre-emptively launched iPhone applications on the assumption that such an application is just a web application running in Safari at a set screen resolution of 480 x 320 pixels. There is no installation process for these applications, and it is likely that these applications will not be able to add their own icon to the iPhone home screen - for the user these applications will be nothing more than a bookmark in their web browser. There is no distribution mechanism, there is no application discovery path and there is no way for a developer to extend or alter any aspect of the iPhone at all. Apple have announced that Core Animation, a cool new technology that allows developers to easily create and manipulate graphics and media on OS X, will be included in the iPhone - but again there is nothing that suggests that this technology will be available to anybody else other than Apple and it’s chosen partners.

One of the most interesting, somewhat controversial and disappointing aspect of development for the iPhone is that there will be no support for either Java or Flash on the iPhone. Flash technology has been the catalyst for some of the best (and worst) experiences and applications on the web - especially around video and other media. I can only think that the reasoning for not including either technology is for Apple to further protect its own applications. Without Flash or Java, you won’t see third party applications that provide any form of rich functionality in terms of music streaming, video streaming or richer interfaces. Such applications would obviously threaten iTunes and the iTunes store - as well as the other formal relationships that Apple have with iPhone launch partners such as Google (YouTube). So developers are not only limited to the web browser, but again are limited to providing applications that can be implemented in only HTML and Javascript alone.

When somebody eventually publishes a hack for the iPhone, and I am sure that it will happen within days, I would hasten to guess that the reaction from Apple to the first citings of Flash on the iPhone, and then music streaming services such as Pandora on the iPhone, will not be very friendly. The same can be said for the first hacks that will allow third-party applications to be installed, with new custom versions of the home screen, the menus, interface and other customizations. I can only imagine that after going to so much trouble to providing an Apple-only environment on their new device that Apple will go to a lot of trouble to protect their closed platform. Just as there are hacks for the iPod such as user-replaceable

If you frame your thinking about this device in terms of the interests of Apple and their intention of driving further usage of their own applications on the device, and in turn further profit, you can see why some of the design decisions were made. Apple know that the mobile device market is a tough one, and that margins in the mobile hardware business are thin and risky - so they see it as having no choice other than to be in total control of both the platform and the user. This is what Verizon and other networks do, and this is what AOL used to do. The potential of the technology in the hands of third party developers is killed in favor of business interests, and it seems that so far with the iPhone most of us are blind to this (and plenty of these people are the same people that chastise Microsoft at the slightest misgiving)

My biggest disappointment is that the same hackers and technologists who cherish, support and have fought for the openness of the web, open standards and open platforms are blinded to the proprietary nature of the iPhone. I can only hope that Apple will eventually come around and open the iPhone up, but this will only happen if it is driven by the users and the development community. Apple have tested our reaction here by coming out at the launch of the iPhone at very extreme closed position - and so far they have won us over, giving them no reason so far as to why they should change their way.

It can be argued that it is well within Apples right to control the iPhone as to how they see fit - after all, they went to all the trouble of developing, manufacturing and releasing the product. But I just can’t buy their public justification for some of their decisions, such as the lack of Flash and Java is said to be because of security issues - the same for not allowing applications. Why would the security circumstances differ just because of a smaller form factor? Just as the mobile world began to show signs of awakening and opening, the iPhone is nothing more than a step backwards. If we don’t accept it on our desktops, then why should we accept it in our handhelds. In the end is up to the users as to which mobile world they want to live in - either one that is dominated by a single company from top to bottom, or the better world of accessible, flexible and free platforms where innovation and development can flourish.