Update: As requested, here’s a demo site.

In preparation for a new CodeIgniter project starting very soon, I prepared a new project template. While CI gives you a great library and framework, out of the “box” it is missing a few things that are necessary for 90% of web applications. Specifically, user registration and authentication, menus, templates, a CSS base, etc. Other frameworks, such as Yii or Django, address this with generators. I think there is some value in creating one’s own. A fine alternative would be to go with another CI-based project framework, especially FUEL CMS, or perhaps PyroCMS.

I am very intrigued by the Sparks initiative also; I think it’s a great idea and a good way to let developers like me pick and choose open libraries, as well as give back. So my first shot at this involves setting up Sparks, integrating a few useful ones, and making a basic site with templates, navigation, registration and login. Everything else will be project-specific, I imagine.

In any event, here’s what I included in my first version:

• CodeIgniter 2.0.2
• Phil Sturgeon’s Asset library
• David McReynolds’ Menu library from Fuel CMS (update: I converted this to a spark too)
• Sparks, with the following sparks installed:
  • ion_auth 1.4.0
  • formbuilder 1.3.0
  • template 1.8.0
• Login controller to run ion_auth, using formbuilder
• Basic CSS and layout files based on HTML5 template
• Static page controller with nice 404 handling
• Dropdown menu 0.7.2 CSS from Live Web Institute

At White Lion, we had use of an in-house framework built on Kohana that made starting a new site very easy. Since I’m back on my own I can’t use their framework for my own projects but I see the value of using a full-blown CMS system as a starter, not just a framework. There are a few CodeIgniter-based CMS projects with some traction, but PyroCMS and Fuel CMS both grabbed my attention. PyroCMS won out because it’s reached 1.2 and module development is very simple, and I’m a fan of Phil Sturgeon.

I have so far tried my hand at developing two modules for PyroCMS: one is a sitemap module which outputs both a human readable and search engine friendly XML sitemap of all pages and blog posts, and the second is a blog importer, mainly for moving my content on this blog from WordPress to PyroCMS. I dutifully submitted both to the add-on store, but both haven’t been accepted after a week or so of sitting around “waiting for approval”. So, temporarily, I will offer both downloads here as a trial: (these are now live on the add-on section, links have been updated)

• Importer Module
• Sitemap Module

If you’re a PyroCMS user or developer, please check them out and let me know what you think on the forum. As for future plugins, I’m thinking of doing a Lucerne search plugin, or a simple Google Checkout/Cart module, or maybe a call-me widget based on Twilio’s API.

I am a freelance web developer, and if you want to hire me to build something for your website, please get in touch.

For two months this spring I had an awesome time working at White Lion in Austin, Texas. Steve Kahle has a fine group of people working for him over there, and there were many interesting projects to complete. I certainly hope I’ll be able to work with them again. Sadly, we couldn’t stay in Austin due to health issues and so we’re back in Wisconsin right now.

One interesting project I spent a few hours on was to see if there was anything to do about the White Lion home page. It is currently an eye-catching Flash image of thumbnails of different clients and case studies. When you hover over one, the thumbnail image moves about while the others fade out. It seemed to me there was nothing about it that specifically required Flash other than the monochrome effect, and even that’s technically possible without Flash. I set about trying to implement the same effect with JavaScript, HTML, and CSS. Here is a static version of my effort.

While it looks and behaves similarly to the current home page, the difference is apparent on every page load after the first, especially if you use the back button. My version is nearly instant while the Flash version always does some reloading. Why? In the current site, the Flash element does not contain any images. Instead, when loaded it always makes another network call to http://wlion.com/home/portfolios, which returns some XML that is then parsed to build the image grid. While this means White Lion can change the home page portfolio listing easily, it also slows down the whole page. The Flash element will not start loading images until after it has itself finished loading, and has retrieved and parsed the protfolio XML.

In my version, the images are just normal IMG tags. If you turn off JavaScript, it still looks and functions similarly, except there’s no movement and the mouse over is a little strange. The advantage is that the images start loading right after the HTML is parsed. On a normal connection, this alone shaves two seconds off the overall load. I believe it would be possible to keep the current design but get the initial page load down to less than 1 second with a few more tricks, such as image spriting and using a CDN host for static assets, or even just using CloudFlare as a reverse proxy. While White Lion decided to keep the Flash element in place for now, certain .htaccess changes also sped things up too, like using GZip and enabling expiration.

The takeaway is that there may be ways to shave precious seconds from your home page load without altering the design. Building a snappy first page is a good investment.

I recently set up and relaunched LoganCreekSoap.com using the BigCommerce store hosting platform. I have to say: e-commerce (and web applications in general) has come a long way in the last decade. The previous platform and setup that they were using before was simply awful for both usability and aesthetically. It really was a shame because they make beautiful handcrafted soap and their store simply didn’t reflect their high quality product.

BigCommerce offered many features that were especially convenient for them to manage as store owners, such as a nice back-end management interface, lots of reports, plug-ins to several third-party systems (like Google Base and Facebook) and a great order fulfillment process flow. As a developer, I was fascinated by the API and started working on a little project to make buying postage a little easier (using Stamps.com’s API), but I was highly disappointed that the BigCommerce API is presently read-only. This is a tiny downside in the vast upside that is BigCommerce. Customers will get many shopping features that they frankly expect, such as seeing order status and history, wish lists, instant shipping and tax estimates, and coupons.

The serious competitor to BigCommerce is Shopify right now. Shopify’s API is better and their app store is highly enticing to me as a developer (nothing like that currently exists for BigCommerce sadly). I couldn’t recommend Shopify for Logan Creek Soap, however, mainly because Shopify takes a percentage of each sale in addition to their monthly fee, while BigCommerce does not. Personally, I intend to find some more local merchants and see if they’d like to let me help them upgrade their stores to BigCommerce. If you’d like my help or advice in setting up your store, please contact me.

I am very pleased that Twilio’s OpenVBX launched today. This is a project that I had the great pleasure of working on last year for several months with the fine folks at Twilio. When I last touched it, I thought it was 99% ready to go and felt a little frustrated that it wasn’t launched sooner.

Now that I see what they’ve been working on in the meantime I understand. A great deal of refinement and documentation has made this product so much better than what it was. They even made a very Apple-esque intro video to describe what it is and why it is a win for businesses. I am installing it now on my server and will update with my impressions soon.

While not yet advertised anywhere, OpenVBX is based on CodeIgniter, a very popular PHP framework. Even if you are more familiar with other frameworks like CakePHP, Kohana, Silverlight, etc., you should find the code easy to understand.

Commenters on this news at HN have been oddly upset over Twilio’s reasonable per-minute charge, which is not new. “Why pay 3¢ a minute when you can roll your own PBX with Asterisk and pay 0.0000000000000001¢ per minute with some VOIP provider?” they ask. Fair enough. But when you go eat a burger for lunch today, ask yourself why you didn’t raise and slaughter your own beef, grind the meat, and fry your own patties? Why pay a farmer, butcher, grocer, and restauranteur to do what you could have done yourself?

By the way, when it the last time someone has successfully used the word “restauranteur” in a sentence?

I have thoroughly enjoyed working with Twilio for the last 6 months or so on an open-source project that will soon be released (can’t wait!). Now that my part in that project has wrapped up, I’ve got some extra time to play around with my own projects and look for my next job. Over on the Twilio support forums, someone recently requested help, and it caught my eye:

I have to conduct a lot of phone interviews every year for my company with fairly standard questions. Would there be a way to use Twilio to manage this in an automated fashion? I don’t want to spend the time actually conducting the interviews but would rather listen to them when I have a chance. [...]
During the call, I would like to have the user enter the # sign or something when they have finished answering the question, to prompt the next question. This way, I could have my 10 standard questions recorded in Mp3, and then have the whole process managed through Twilio ideally where i enter the interviewee’s phone number and a time for the call ideally.

Someone posted a link there that lead me to this service: VoiceScreener which starts at $169/month. Yikes! Twilio, on the other hand, is 3¢/minute and $5/month/phone number. I got to thinking: this is incredibly easy to set up with Twilio’s API; why not just whip it up? So I did.

Here’s a demo where you can try a sample phone interview and here’s where you can download the code. To install and customize it, just follow these directions:

1. Unzip the file to a directory on a web server that runs PHP 5
2. Edit the file inc/config.php with your own Twilio authentication credentials and the email address that will receive reports.
3. I have included my sample MP3 files in the files directory, but you should replace them with your own. The file names should be self explanatory: intro.mp3, thanks.mp3, conclusion.mp3, and question_XX.mp3, where XX is a two-digit number.

As it is written now, this works without any database at all because we can use Twilio’s REST API to query the call details and list of associated recordings. This project is more of a starter and proof of concept, although for simple interview scenarios it would work just fine. Other possible upgrades (some requiring a database) could be:

• Account for unanswered questions in the report
• Allow interviewers to hang up in the middle of the interview and call back later to pick up where they left off
• Allow the interviewer to review their answer and re-record
• Allow for question branching, where the user enters a touch-tone response for yes/no on some answers and is asked only relevant questions from then on
• Set up multiple types of interviews
• Show the text of the question in the interviewer’s web browser  while they are listening to it on the phone using AJAX
• Make the interview conclusion a menu, as in: “To hear more about the company, press 1. To hear more about this position, press 2…”

If you are interested in having me work on this for you, please contact me and I’d be happy to help you out. If you have any suggestions or improvements, please send them my way too.

Update 6/1/11: WebRTC looks very promising.

Update 11/05/09: Looks like this might be possible in Flash 10.1! (via)

I’ve been working on a project recently that would really benefit from the ability to record audio in the browser and upload it to a web server. This would be a better experience than an “upload your sound file” dialog, where the user would have to record themselves in another app, find the file, and upload it. Here’s how I imagine a seamless audio capture experience would work inside a web application. There would be a small widget in the page that would look like this (icons from FamFamFam.com):

When the widget first shows, only the record button would be enabled. The left side has a line-level indicator that reflects the sound level at all times.

While recording, the recording button changes to a stop button.

Once the recording is stopped, the stop button changes back to a record button. The play and upload buttons are enabled. Clicking upload at this point saves the file to the server, and the whole control would change to an upload progress bar. Clicking record here starts a new recording and erases the old one.

During playback, the record button toggles to a stop button again and the play button toggles to a pause button. The line level reflects the playback volume. It is possible that you don’t want to allow upload during playback, but I’m not sure.

I want the control to be slightly customizable for the web developer who’s embedding it. He/she should be able to specify a maximum recording length and a URL where the sound file will be uploaded, or perhaps disable playback and upload immediately, that sort of thing. Maybe the control itself is actually a 1×1 SWF that has methods and events for play, stop, record, and so on, and the controls would be standard HTML wired up with JavaScript.

I assumed that other, smarter people have already tackled this problem and there is some Flash control I could embed in my page and a little PHP script or something to handle saving the file from a POST. If you’ve ever searched for this sort of thing before, you already know that it does not yet exist, at least not like I have described it.  Microsoft Silverlight doesn’t yet have microphone access, and I can’t find a Java applet that does this thing either. (I found this one, hmmm. Wish it was free/Flash). There are several very nice Flash playback scripts (SoundManager 2, Yahoo Media Player, JW FLV Media Player, JPlayer, NiftyPlayer, WordPress Audio Player, etc. etc. etc.), but nothing that handles creating uploading and saving recordings. Yet.

Flash allows for audio streaming from the client microphone, but in order to capture the stream you need a server that understands Adobe’s streaming format called RTMP and can do something with it, most likely some kind of crazy-expensive Flash Media Server or the open source Red 5 server. Neither solution is exactly straightforward or doable on a simple web host; it’s huge overkill for this one thing. If your project is meant to be downloaded and run on other servers (like WordPress and my new project) this kind of requirement is deal breaker. Besides, it doesn’t really allow you to preview the recording before it’s uploaded. Frustration with Adobe’s “closed” microphone interface has caused some Flash developers to start the Open Mic initiative at GetMicrophone.com. I wish them luck with their effort.

I recently asked my friend Joey Betz (a very talented Flash programmer who works for Armor Games) if what I describe above is possible. He said that up until Flash 9 it is impossible, but he could imagine that with new access to microphone sound data in Flash 10, it may be possible to get the byte stream of the sound and base-64 encode and post it to a web script. I guess nobody has the combined need, interest and Flash/PHP chops to make it happen – I have all but the Flash bit. I’m not sure anyone knows the actual format of the byte stream: is it just raw sound like a WAV or what? Can you turn around and replay that byte stream through Flash? How do you turn it into an MP3/OGG stream? Even if the byte stream is something odd or uncompressed, perhaps the PHP script could do the heavy lifting to re-encode it; many servers also have FFMPEG or Mencode or something like them installed. Alternatively, what about a web service with an API where I can stream data to a Flash Media Server or something like it and get back a URL to the resulting MP3? I just don’t think recording a few seconds of sound and saving it to a file requires a separate server daemon and all that entails. Real-time voice/video chat — sure, but that’s not what I’m after.

Unless someone can convince me that this is still not possible, I will be happy to offer some of my time and server bandwidth, hosted now at MediaTemple, to help a creative Flash programmer get the server-side bit of this working. I’d like to release this widget/script as open source to the web community. I am looking especially to the creators of any of the aforementioned Flash playback scripts. Can you help make this happen?

I had an idea for making open geographic tags over the weekend so I hacked out a quick proof of concept in JavaScript. If you hate reading, you can skip to my demo here and play with it now.

You may have read the recent noise about URL shortening services being “the herpes of the web“. Please. I think this comes down to tagging being a closed system, but it’s not easy to fix and I doubt these kinds of things will ever go away.

It got me thinking, though: links to geographic information could benefit from shortening, and selectively loosing information could even be an advantage. Latitude and longitude is expressed as a pair of numbers, traditionally divided up by degrees, minutes, and seconds, but digitally expressed as floating numbers between -90/90 and -180/180 respectively. For example, a beach near my home is at 33.54240,-117.78559. For example, say I wanted to tell my friends that I’m at the beach, and my Twitter client could access my GPS and append my location to the message (yes I know about Twinkle). That’s 19 characters out of 140 and it’s very public. Also it’s a single point, not a whole area, and how do I search for other people near me?

My idea is to create a new kind of geographic code that specifies areas, not points, and is constructed in a text-search friendly way. Here’s how it works. First divide the world into a grid of 36 x 36, or 1,296 squares. Assign each row a number or letter using 0-9 and A-Z, that’s why 36. If we specify one of these grids by coordinates, such as “o6″ for example, that would specify a rather large part of the earth, as you can see below. If we were then to reference a point using a further 36 x 36 subdivision of this grid, we could specify an area even further, such as “o6p7″, which would be smaller still, and so on and so forth, all the way down to “o6p7iy6z”, the  rough equivalent of the 19-character coordinates above in only 8 characters.

The more specific you want to be about your location, the more subdivisions your code would include, although using base 36, four levels is about the size of a house, three levels is a few blocks, two levels is about an hour’s drive, and one level is a day’s drive. That sizing feels right to me, but perhaps if this ever gets used people will standardize on different subdivisions, such as base 16 or base 24. Anyway, if you’re sharing your location but want to protect your privacy, leaving your location general and loosing a little specificity is a good thing. The subdivisions function like decimal places in that case. “o6″ means somewhere in SoCal, while “o6p7iy6z” means a certain spot by the volleyball nets on Main Beach in Laguna Beach, California.

When looking for proximity, you can see at a glance if a code is near you. In my case, I wouldn’t bother with things that don’t start with “o6p7″, although I might look at “06p8″ or “o6q7″, which are adjacent. A text search engine could find nearby items tags a wildcard search pattern (“06p7*”).

One other major benefit of a protocol like this is that it would be open. It’s not perfect; you may live close to the boundary of a big grid, and there’s an up-front hassle in figuring out exactly what grid you’re presently in unless your device can calculate your geo tag for you. Once in place, no single company would be the gatekeeper for the information that makes the tags meaningful, as in the case of URL shorteners. A unified location tag protocol could open up many interesting mashups between location aware devices and services, not just Twitter et al.

If someone’s already done something like this, then I wasn’t able to find it and I apologize, please let me know. If you want to fix it up and build upon this idea, by all means go for it but please link back here. Please view the source of the HTML demo file for implementation details and send any errors or suggestions. I think it’s not a perfect square because of the distortion from the map projection, but I could have flubbed something too.

Update 8/20: tr.im has since threatened to shut down (then not), and bit.ly has proposed the FDIC of shortners. I look back on my post below and concede that the hashing thing will never happen, but the central backup surely should occur, and 301works.org is the best chance it will come about. Any service that claims protecting your “privacy” prevents them from backing up their link archive with a 3rd party is lying or misunderstands the purpose of a central repository and shouldn’t be trusted with your links anyway.

The latest brouhaha over URL shorteners is overblown. Are they evil or unnecessary? Who cares? They’re not going away. Proclaiming the death of “unnecessary” institutions is a tired cliché. I’m all for self-documenting pretty URLs like the next guy, but come on.

I feel that the core argument boils down to discomfort with the opaqueness of the the whole thing. Where does a shortened link go? You don’t know until you’re there. What kind of permanence does the shortening service itself have? This kind of worry is an echo of the greater distrustful zeitgeist though – if AIG and GM can teeter on the edge, what hope can we have for TinyURL? That’s probably how Philistine foot soldiers felt when Goliath fell.

It appears that many shorteners use an auto-incrementing database key to store link references. Thus the first link is http://example.com/1, and the second is http://example.com/2, and by using all letters of the alphabet (using base 36) then link number 999,999,999 will only be http://example.com/GJDGXR, six extra characters. Since they’re not all using the same database, then the true link for http://shortenerA.com/GJDGXR does not relate to http://shortenerB.com/GJDGXR. If Shortner A looses their database (it happens) then goodbye all links. Thus, shortening services are opaque and creaky by design, namely big-honkin’ private central database.

If the digerati can’t abide secrets, then they should gather up their pitchforks and demand that their favorite shortening and bookmarking services transition away from private, proprietary incrementing keys to an open URL shortening hashing scheme and distributed URL repository for backup. How would this work? Well, Netcraft reported that there were 224,749,695 servers on the Internet in March 2009. Let’s just round that up to 250 million and say that each site has an average of 200 URLs, which is 50 billion, or MYWPIWW — just seven characters — in base-36. You could normalize and hash any URL through a 40-bit hashing function and format that number to a base-36 string. All participating open shortening services would have to use the same hashing method and synchronize their URLs with a central repository. By using a hash function, each service could defer repository synchronization and use it as an insurance policy. The central repository would be queried only when a hash doesn’t exist locally such as when a competing service goes dark or offline. Storage of 5 billion URL records in the repository (assuming each record is 1kb) would require 4.65 TB of storage, not quite doable with Amazon SimpleDB just yet. Nevertheless, I can easily imagine two guys hacking up a robust shortened URL repository at this scale over a weekend using tools like Amazon’s EC2, SDB, and SQS and charging per-use fees to the services for a syncing API. The repository has the added benefit of being a neutral third party that can validate and lengthen links for security.

I have no illusions that this system will ever actually be built, or if it is, that it will be used widely. You’re welcome anyway.

Here is a quick rundown of how my recently launched site ThisLineIsSecure.com was built. First of all, it would not have been possible without the Twilio API. Besides that:

• Written in PHP 5 using the Kohana Framework, the able successor to CodeIgniter.
• Hosted at MediaTemple using the Grid Service (like this blog).
• Developed on my iMac using Active State’s Komodo Edit.
• Graphics done using Pixelmator, stock art from Fotolia.
• Web Template adapted from Impress 05 Nuvio.
• Some icons taken from the Function Icon Set.
• Using the Standalone version of the WordPress Audio Player.
• Google Checkout & Google Analytics for payments.
• Version control and issue tracking via Unfuddle’s free version, which is enough for me. I have used this on huge projects, though, it’s very good. Think Basecamp + Beanstalk.
• SFTP uploads via the finest Mac shareware FTP client available, Cyberduck. The “synchronize” option is golden.

I need to tally up my expenses, but besides my time I’m probably out less than $200 for this whole thing so far. Unbelievable. I think that the Kohana project, Nuvio, the WordPress Audio Player, and Cyberduck each deserve at least a $20 donation, which will be forthcoming.