PyDev and PyAMF; force them to play nice

Short one: I use PyDev for my google GAE efforts.

I used PyAMF as detailed in the PyAMF Google App Engine tutorial (which is braindead easy; just copy the pyamf directory into the base of your GAE project). When added to a vanilla GAE project created with the GAE SDK tool, it worked fine.

But then I created a project in Eclipse with PyDev; it's always worked. But I dumped in the pyamf directory and, although auto complete worked and I could "click into" the pyamf classes after importing, the app wouldn't run; import errors (no such module). I knew it was a pythonpath problem of some kind, but I couldn't figure it out, and I tried everything.

I eventually did this, and it worked; I retried it several times, it seems to be the fix for this sort of thing, possibly any such issue related to using a PyDev project.

Create project in PyDev, but do NOT configure python path (use the "configure later" option), and do NOT configure src directory; just use the base project path. Do all other configs as normal (your GOOGLE_APP_ENGINE config and so on, as if you were doing a usual project).

Then drop your PyAMF directory into the root of your PyDev project folder (remember, you don't have a src directory), doctor up your main.py to use the WebAppGateway and so on, and it finally sees all the imports and works as expected.

No idea what's going on, I'd say there's something going on in the way PyDev interprets paths vs. the way many libraries like PyAmf are set up to do their imports. I'm sure there is an elaborate answer to this involving site-directories and so on, but all I wanted was for my PyDev project to see my PyAMF modules properly, and the above steps, which simplify the PyDev project config drastically, seems to work.

Hope that saves sombody some effort, it killed half a day for me.

Clouds: EC2 and GAE in the Indie Dev world.

"Cloud computing" is a big bleeding-edge phrase these days.

In a way, all it really means, is that there's a processing and storage service out there, on which you can build and run software. You don't have to buy or maintain the machines, and for the most part, you don't have to install any of the base software (like operating systems, core services like data and cacheing, etc.). It makes app dev a lot easier; you register an account with your cloud provider, they allocate you a CPU and storage budget, and provide a way for you upload your software to run. This is of course primarily Internet-based apps, anything from a website to a full-on financial RIA.

In another way, it also means, that you now work under the restrictions of that cloud provider. If they don't want you running certain classes in their cloud infrastructure, they blacklist it; you have to find an alternative. Such restrictions usually involve IO operations like file writes, security and encryption, and other general black hat sort of stuff.

The trick is to find the balance; a provider that gives you the flexibility you want, without making it just as onerous as actually owning and maintaining the machines (things like auto backups and data replication, etc.). Based on what you need, you go towards either end of that balance.

I've been working with cloud computing as an app developer for some time now. From a developer's perspective, it's all pretty much the same; "what technology, what APIs/frameworks, where do I put it, what basic server admin do I have to perform." So while I heard a lot of people talk about "the cloud" as a mysterious and nebulous thing, I always just thought of it as another server out there to run my cruft.

It's evolving though; it's becoming RIDICULOUSLY easy to put a high end app out there, relative to the difficulty of even a few years ago.

The two clouds I use these days:

- EC2 (Amazon). Couples with their S3 (simple storage) service, provides a way to actually spin up remote machines of almost any configuration. For example, Amazon has a Linux base called a "micro instance", that you can spin up and access via it's IP address just like any other remote server. You log in as ec2-user with your public/private key, switch to root (you can not log in as root), and SSH all day. Usually you have to employ yum a lot, to install apache, FTP, and what have you, and do a lot of RPM gets to fill in some blanks, but they do a good job of making pretty much everything for a basic server confg available.

Admin is simple via the EC2/S3 web interfaces. You can spin up buckets for storage, start/stop instances (which starts/stops billing...you only pay for used CPU and storage space, if your instances are all shut down then all you pay for is storage), make and edit security groups, generate .pems for your keys, and so on. And it's CHEAP. As a dev, I spin up a micro instance, install apache, tomcat, whatever, check the python and perl installs, and am pretty much ready to stage anything. If I shut down the instance when I'm not using it (so say I leave it running 8 hours a day), my bill at the end of the month is literally a few bucks.

Scaling is up to you. If your micro instance gets slammed, it'll bork. There are services out there that can do this for you; if you're trying to get out of the hardware end of things by getting into cloud computing, a service like this is essential, or you're just going to end up hiring your network and server guys all over again. You can of course also manage it yourself, the EC2 admin tools provide APIs and such to get it done.

EC2 I like it a lot. I actually canceled all my traditional server hosting, which was costing me in excess of $200 a month just to be able to stage apps for clients, and now it costs me maybe $10 or so. It's also very flexible; basically, there's no difference to me, it's easy to think you actually have a real server out there, when all you actually have is a chunk of the CPU clay pulled off the big CPU clay ball; you configure that chunk however you want, be it Windows, Linux, Unix, web server, app server, whatever. There are some security and storage restrictions, but nothing I've ever run into in day-to-day dev.

- Google App Engine (GAE)

I've been writing code as a subcontractor for Google projects these days, (YouTube Town Hall, that sort of thing), and use the GAE for my backends.

GAE goes the other way from EC2; where EC2 is heavy on flexibility in favor of adding some complexity (like needing public private keys for logins, configuring security groups, having to install things on your instances, monitoring the instances carefully), GAE provides a prescribed way of developing your apps. You can use either Python or Java (unlike the EC2, in which you can use and dev technology I've ever seen). Access for devs is tied to a gmail account. You just sign up for the GAE, get an account/app key, and can deploy 10 apps, utilizing a decent amount of CPU and storage, for free (for now anyway).

Regarding "Java or Python for the GAE", I've used both; in general, I lean towards Python unless Python makes it difficult, and it often does. I have very unstable results with PyAMF on the GAE, but know I can get AMF running in Java on the GAE, so when I need AMF, I use Java. Also, the Objectify data framework for GAE (Java based) is very powerful and well written, it makes using the GAE datastore pretty straightforward.

You don't have the tactile sensation EC2 gives you, which is of "solid server". You deploy your code as an upload (like FTPing), and it compiles, validates, and deploys it. You then access the functioning app online at [yourapp].appspot.com. Servlets, services, and all that, provided they made the blacklist cut, will all run as expected. You have an admin interface that tells you how much CPU and storage your app is using, and you can see things like what's in the task queue, inspect your data store, see usage stats, and whatnot. The data service is in the form of an object store (not a relational DB), there's memcache service, a task queue (configurable queues too), cron capability, a bulk data uploader/downloader (which is an admin tool, not an app tool) and some messaging capability. Everything you need to build out a fairly heavy duty app.

However, those services are what you have to use if you want to do your processing with GAE resources. Unless there are arrangements different than the default one I have, what you see is what you get. Things like file read/writes and such are also much more heavily restricted in the GAE, putting some solid walls around your app and what it can and can't do. If you're inventive, you can generally use the tools they provide to meet the need.

Scaling is braindead easy in the GAE; you don't think about it really. You just up your budget for CPU/Storage, and you get more CPU time. If your app hits a threshold, the GAE also automatically spins up more "Instances" so that your app can load balance (sort of). You don't worry about it at all, as long as you have the budget, you're good. Even under very heavy load (like hundreds of thousands of hits) I've not seen this budget exceed 60 bucks in a day, and that's VERY unusual, typically it's a few dollars for a reasonably busy app that isn't doing huge background processing jobs (which I would offload to the EC2...)

Because the GAE is much more prescribed, all you have to do is get that prescribed model, then it's pretty EASY. You fall into repeatable patterns because there aren't multiple ways to do things like data writes; you use Objectify on the GAE datastore API, or use JDO and beware blacklisted classes. It's also very easy to admin; you go to your GAE interface and can inspect the data store, task queues, see how much CPU and storage your app is using, etc. There are some deeper tools too, like the ability to use custom domains instead of appspot ones, but I haven't needed anything like that yet.

Which do I like better? Hard to say, because I've combined them for a couple of apps. I use the EC2 to spin up instances on which, say, I need to do a lot of scheduled processing, like image or video download/processing. If I have particular requirements from a client for data storage or web serving, EC2 is also the way to go. But for deploying and serving general web application, even ones with some pretty slick data and task requirements, the GAE, as long as python or java is ok with you, is a great, easy, and even fun, way to get the job done.

Either way, both save you a lot of money and headache. Naturally, the heavier your use, the more money and headaches you'll expend, but for general app dev, particular for testing and staging, and even for long-run apps if you know what you're doing, dump your old hard boxes and use the clouds.

As always, thanks for visiting.

Parsing Params from a UI in PHP

I've seen a lot of this out there, and seen a variety of scripts that...well...just don't work. People recommend plugins, frameworks, all that, just for the simple task of some basic params parsing.

To be frank, as I go on using Zend framework, I find it hard to believe the amount of effort people will put into searching for pre-made solutions for things that are really pretty simple. I suspect this has to do with the fact that "framework" is misinterpreted as "does everything for you", which leads a lot of what I call "script hackers" to pursue them, when in fact using frameworks is a more advanced programming undertaking that, imho, separates the men from the boys, as it were. I understand that situationally a lot of that might be useful, but for me, when it comes to basic operations such as this, I prefer to use the features of the language without additional bolt-on libs, mods, and what have you.

Here's the code I use, which has worked consistently well, in a little util class. If you have other character or encoding concerns, just bolt them into the processing before you hit the foreach loop.

public static function parseURIParams ( $uri )
{
 $params = parse_url ( $uri, PHP_URL_QUERY );
 $values = explode ( '&', $params );
 $finalParamsArray = array ( );
 
 foreach ( $values as $str )
 {
  $keyVal = explode ( '=', $str );
  $finalParamsArray [ $keyVal [ 0 ] ] = $keyVal [ 1 ];
 }
  
 return $finalParamsArray;
}

DataGrid Reminder: Item Renderers Don't Trigger Events Until They're Visible

Ran into this recently; no code, since that's not really necessary. But this info is pretty valuable imho. I've seen tons of info on the net in all the forums that work around this every which way without every actually stating the precise issue.

Say you've set up the following:

- A datagrid with a dataprovider (say, XML).
- One of the columns in the datagrid has a custom renderer.
- You need to update a model (say, processed XML you will send to a service after the user is done with the grid) with some data that is automatically populated in that renderer's UI field (like a default value).

For instance, say you have this:

<item>

<itemdata>Hello</itemdata>

</item>

You use this as your dataprovider. Your custom renderer has a textfield. You write Hello into the textfield on dataChange. Nice nice so far.

Now, when that textfield changes, you want to update a model of XML, or trigger some other event, whatever. You add the textfields change handler, you see it work, you breakpoint the code that updates the model, and you see the info added. Nice.

Then you say, "let me blow it out to 20 rows". You add them all, you breakpoint the model code, and it works...up to a point. For some reason, 7 or so of the pieces of data you want in the model are there, but then it just stops, as if the data is getting skipped or just not sent.

You fiddle with it...sometimes 7, sometimes, 5, sometimes all 10. WTF is going on?

If you know the answer already, raise your hand...but I bet you don't (I looked and asked around for a week before figuring it out).

Drum roll...

...datagrids only render visible rows. If you had to scroll to see any of your rows, then those renderer's events never fired (like it's textfield change event). Which means that the events never dispatch, so your model, or whatever, never gets the message.

So, if you're counting on events from a renderer events to update a model or trigger some other action when all the data is initially loaded into the datagrid based on a value set in a renderer, don't dispatch the events from the renderers, because if they aren't visible, they'll never send their messages.

Perhaps a datagrid creationPolicy of "all" would be useful under certain circumstances.

As always, thanks for visiting.

Passing Simple QueryString Data from SWF to SWF using SWFLoader

Lot of posts out there with people trying to get around this one. It's a pain; under certain (most) conditions, if you try to simply load a swf into a swfLoader, and pass parameters by appending the path to the swf, when you test the parameters in the loaded swf, you get nada.

Ugh.

Easy workaround; use this in the LOADED swf (so in the code of the swf you loaded):

Application.application.loaderInfo.loaderURL

That reveals the entire path the loader of that swf used. After that, parse out your values and off you go. Not as nice as if they appeared as neatly packaged object properties, but it works, and parsing a querystring is 101 stuff anyway.

Easy one, but effective.

As always, thanks for visiting.

Adobe CS5 iPhone building is "*Subject to Apple's current requirements and approval."

If you read my last post, you know that Apple altered their licensing agreement for applications. Not only is apple telling you how you can distribute them, but now they are telling you how you can build them.

Here's the meat of the matter:

"3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited)."

It's corporate control at it's worst. Apple has surpassed Microsoft in terms of "evil empire". Microsoft may be berserk with their licensing pricing and enforcement, but one thing is true, they NEVER told me that I couldn't use my computer whatever way I wanted as far as a developer. As long as the binary output runs, I can install it, for FREE. Apple, however, has decided that's not enough money or control. Now they want to specify the language you write in, how it should be compiled, and all the rest of it, AND get a fee JUST so you can put an app on an iPhone, iPod Touch, or iPad. The more I look at it, the more I see corporate, hysterical paranoia. Apple simply wants to own and control all content distribution on the web.

Apple = Open Web? No, they do not. They = closed devices supporting the technologies they choose, the way they specify. They are doing evil; they could learn a lot from Google.

What evil?

The change has affected numerous small development shops that were counting on Adobe's Release of Flash CS5, as well as shops working on compilers for various dev technologies. Apple is taking every step they can to keep anything other than Apple licensed developers, and Apple licensed tools, out of the realm of Apple mobile development, in particular, the petulant snubbing of Adobe's Flash. They offer all kinds of weak, easily repudiated reasons for not allowing the technology to be installed in their Safari browser on the iPad and the iPhone, or as a runtime that can support Air apps. I won't rehash these, other than to say, if a $250 netbook can run Flash and Air, then there's no reason in the universe that the iPad shouldn't be able to, unless all it really comes down to is that the iPad is a glorified iPod Touch without the camera. I mean...no USB port...no Flash...no multitasking yet...come on. Has Apple forgotten that technology isn't just about designing cool looking things?

Anyway, Adobe has added this little quip to the pre-release announcement of the CS5 Master Suite, due for release in mid-May:

"Publish content virtually anywhere, using Adobe AIR® for desktop applications and mobile platforms including the iPhone®*, or Adobe Flash Player for browser-based experiences.
*Subject to Apple's current requirements and approval."

Notice that last line. This means that, Adobe may give you a tool that can build iPhone apps from Flash/ActionScript...but you may never be able to use it. All Apple has to do is keep modifying the license agreement to shut Adobe out forever, and as they clearly show by the new 3.3.1 paragraph, they're more than willing to do it.

For shame Apple. You're supposed to enfranchise developers, let us create tools to work with your platform. But instead, you react by putting more mortar on the walls around your mobile platform, paid for by developers that just want to put an app on an iPad.

HP Slate for me (google it), and depending on how Android 2 goes over the remainder of my iPhone contract, it's a distinct possibility I'll abandon Apple's mobile platform altogether. My whole perception of what I see when I look at my iPhone has changed, and I don't like it. Sure, this may only matter to developers, but developers build the apps you love. Historically, where the developer goes, the consumer follows. If Apple keeps heading this way, it's just a matter of time until people start jumping ship.

Is this all written in stone? Hard to say. But it is indeed written, at least for now.

Apple tightens control over development...again.

Take a look at this: Before today, the Apple License Agreement for apps read like this:

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs.

As of today, it looks like this:

3.3.1 — Applications may only use Documented APIs in the manner prescribed by Apple and must not use or call any private APIs. Applications must be originally written in Objective-C, C, C++, or JavaScript as executed by the iPhone OS WebKit engine, and only code written in C, C++, and Objective-C may compile and directly link against the Documented APIs (e.g., Applications that link to Documented APIs through an intermediary translation or compatibility layer or tool are prohibited).

Now, I don't think this necessarily means the Flash CS5 is torched (if you didn't know Flash CS5 has the ability to compile ActionScript apps to run on iPhone). The only thing it MUST do is, it must appear to the external eye that the app was written in C (objective, C++, etc.). If there is a compiled binary that speaks to the iPhone API through an intermediary layer, it's out of license.

Whatever the case, this is going too far. Not only does Apple want to specify how you can distribute your apps, they want to tell you exactly how to write and compile them. I'm reasonably confident this won't stop Adobe, as they wouldn't have gone this far without understanding Apple's probable reaction to their iPhone compiling tools, but it will certainly torch a number of efforts underway to get iPhone dev into general development, like Java, etc; I know for sure the .Net->to->iPhone effort is now out of license, as it uses an intermediary layer (or did anyway).

C'mon Apple. You've got your distro lock, for pete's sake if home engineers can come up with another way to put it on wax, and it passes your already existing restrictions, back the hell off.