November 2010 Dublin GTUG meet

Last night, we had the November 2010 Dublin GTUG meet. This is a short report on the meet.

As the GTUG has been evolving, we’ve been keen to try to get the sessions more focused on one technology, rather than having quite a mixed bag – this makes it easier for a potential attendee to decide if they are interested or not in the session. Last night was really the first session operating in this mode – the session was specifically focused on the Google App Marketplace.

The session comprised of two longer talks – one given by Claudio Cherubino of Google Developer Relations and one given by Morgan Lynch of Yendo.com. The two talks complemented each other very nicely: Claudio opened the proceedings by discussing Apps Marketplace in general and Morgan gave some specific experience using App Marketplace from a business and application developer perspective.

Claudio started by describing what App Marketplace actually is, how it relates to Google Apps and the usage of the Google Apps platform. He then described the process of developing an app for the Marketplace, what the integration points are, what technologies underpin the App Marketplace and how it works in general. He then gave some more specific example of how easy it is to develop a web-based voicemail app based on Twilio, how the workflow operates and gave some snippets of PHP code to show how to obtain the user’s login identifiers (via OpenID) and authorization to access their data (based on OAuth). Finally, Claudio talked a little about the new Billing API which will be offered via Checkout and will make the job of handling payment for App Marketplace apps much simpler. All in all, the demo application that he showed was put together with about 100 lines of code – it was remarkably powerful for the little work required to develop it, which really shows up the power of the platform.

Morgan Lynch from Yendo then gave his story. Yendo is a small business focused on providing cloud based accounting applications; Morgan had some experience in this space and thought that the offerings out there were typically quite antiquated and there was an opportunity to develop a slick cloud based application which could be offered as a SaaS proposition. Yendo originally developed their solution as a standalone web based offering, however, when App Marketplace was announced they decided to integrate with it. Morgan reported that the integration – with their .Net based application – was a very simple process indeed, taking only a matter of days. Further, the Marketplace has global reach and it enabled Morgan to get access to customers in very diverse, remote places – something that would most likely not have happened without the Marketplace channel. Marketplace gave Morgan access to a large amount of users and his targets for users on the system increased dramatically as a result.

There were some interesting questions posed to both Claudio and Morgan. Some of the questions were:

• how international is Marketplace – not very, it’s still English, but they are aware that this is an issue
• what types of users does Yendo get through Marketplace – mainly v small organizations, often sole traders (and often in jobs that you wouldn’t expect to be signed up to Google Apps)
• what kind of scrutiny does Google give apps sold through Marketplace – v little, they rely on the users of the apps to perform their own due diligence and they have some commenting mechanisms which should show up poor quality apps quite quickly

The only other order of business was to decide if there will be a meeting in December to do some coding. A show of hands convinced us that there is enough interest to give this a go – so this will be organized and details will be circulated. Everyone must get their thinking caps on for cool, but simple, apps to develop!

List of Tech Groups in Ireland

I couldn’t find a list of tech groups in Ireland. EI does have a calendar of events, which is great, but I just wanted a list of groups, rather than a list of events.

I started to pull together some of the info on this page – all pointers and contributions welcome.

 

GAE, App Marketplace, Python and Java

I spent some part of the weekend working on an app which was hosted on Google App Engine (GAE) and hooks into the Google App Marketplace – I learnt a bit about how these systems work together and thought I’d share.

There are some issues with the choice of development languages for an App Marketplace app hosted on GAE – GAE supports Python and Java and App Marketplace currently offers supports for Ruby, Java, C# and PHP. In principle, it is possible for App Marketplace to work with other languages, as it is really just a set of web APIs; however, it could be a bit of work to assemble the right set of libraries to make this work properly.

I did spend a bit of time before looking at using the Python variant of GAE (GAE/P) to develop an app which runs on the Marketplace. However, I found that there are some known problems with Single Sign On for App Marketplace using the Python in AppEngine. For this reason, we chose to use Java in GAE (GAE/J) for the solution.

In my naivety, I had assumed that the GAE/J would be very similar to GAE/P and hence my experience with the latter would make it relatively easy to get a GAE/J app up and running quickly. To my horror, I found that GAE/J is based on Java servlets and is a big world of pain when compared with the ease of GAE/P.

We did make some progress on the concept, but we found that there was a small outfit based in Mountain View doing essentially the same thing – didn’t do the homework properly. We shifted emphasis to produce what is really quite a nice concept – it’s no longer focused on the Marketplace, so we’re going to ditch the Java variant and start again with Python.

Key takeaways for me from this episode:

• GAE/J is very much more complex than GAE/P – you’d need good reasons to use the former;
• if you have designs on an App Marketplace hosted app and you haven’t rolled a line of code yet and time is not of the utmost urgency, I’d advise hanging on until there is proper GAE/P support for Marketplace – it shouldn’t be so long

If you’re interested in Marketplace, are hanging around Dublin and free on Nov 30th 2010, you could pop in to the local GTUG for more info on it.

Update: I’m told Google has fixed up this problem in AppEngine 1.4.0, so GAE/P now plays nice with Marketplace. Haven’t tested it as yet, but I’m assuming they know what they’re talking about.

October 2010 Dublin GTUG

Another interesting night at the Dublin GTUG. We had a very full house last night, what with about 50 folks in attendance – we made an effort to reach out more to students and there was an excellent response.

The evening started with Nick Johnson giving an overview of how Google built App Engine – what problem it was intended to solve, ie making it easy to scale web apps, and what solutions Google arrived at. Nick talked about the different components of App Engine and talked about the how the Datastore works in a bit of detail. There were quite a few questions from the crowd, ranging from issues pertaining to video serving via App Engine (me!), to costs of App Engine, to standardization of App Engine interfaces and Google support of App Engine going forward. A very interesting discussion indeed. Slides here.

Following the break, there were two short talks. The first one, in a series we’re calling Frontline Dispatches – where folks share their war stories – focused on the use of the Maps JS API v2/v3 in the hittheroad.ie application. Fintan Fairmichael talked about how they had used Maps to support their routing application, problems they had, eg issues with street name autocompletion, work involved in moving from v2 to v3 of the API, etc. Slides here.

The second of the short talks was in the Student Showcase series, in which a student showcases their work. Note that the work does not have to be complete (although we do feel that it is important to have something to demo!). Mark Ahern from UCD Computer Science showed his work to date on realizing a web-accessible EC2-hosted Android emulator for remote testing of apps. Slides to be supplied.

I’m told the usual suspects went off to the Schoolhouse for a swift half, but I couldn’t make it on this occasion.

We’re hoping to have a talk on App Marketplace next month – stay tuned to the group for more info.

 

JQuery, JSON and GAE…and then a little REST

Working with JQuery, JSON and GAE proved to be a little less plug and play than I had envisaged. I’m working on a small project in which I’m using client side JQuery to make Ajax calls to a GAE backend – I’m using JSON as the lingua franca between front and back ends, for no other reason than that I like it better.

My requirements to date have been:

• Client-side JQuery makes Ajax call to backend, backend to return list of objects from data store;
• Backend makes REST interface available to provide flexible access to data store

Some issue arose in each of the above which I’m noting here.

Getting GAE to return a list of objects serialized into JSON was not so difficult. There is a discussion on stackoverflow which focuses on exactly this issue – this discussion provides a JSON serializer for GqlQuery objects which can be dropped into the GAE app and imported. I did have to extend the serializer as it doesn’t have great coverage for all data types: in my case, I had to add handlers for a date.date type and a db.Query type. The file should probably be put on github to allow for anyone to add extensions to it.

Getting the REST stuff working was quite a bit more complicated. I chose to use the appengine-rest-server library as it seemed to have been around for a while, have pretty good functionality – JSON support in particular – and it is seeing a significant amount of downloads from code.google.com.

The library is easy to use – just create a rest directory in the project, drop the __init.py__ file into it and import into the main project. It requires a few configuration parameters to be set, eg the endpoint which handles the REST – I did run afoul of some GAE persistency issues with this – the parameters need to be set in code that only runs once in a given run-time.

Once I had the server running, retrieving XML from it proved simple. I encountered two further problems retrieving JSON from it. Firstly, the library expects the HTTP request header to include an Accept field which is exactly ‘application/json’. Setting this was a bit difficult – JQuery does provide for this using either the ajaxSetup() call or the beforeSend: parameter in the ajax() call. However, browsers do not fully respect what they are instructed to do, which makes the process more complicated.

The second, related, problem arose in the way the REST library deals with Accept field. Its MIME type pattern matching algorithm has particular behaviour, which means that an Accept field of the form ‘application/json, */*’ is interpreted as requesting XML rather than JSON. This seemed a little surprising to me, and I made some small modifications to the library to just return JSON, as this was what I was most interested in.

Having resolved those issues, the basic JSON plumbing seemed to be largely working as one would expect. The process seemed a little more niggly than I had expected, which leads me to believe that there aren’t lots of folk out there who are using these technologies together right now.

Simple Javascript HTTP Request Generator

I wanted to have a simple HTTP request generator for some work that I’m doing. I did not want to have the HTTP generated by a standalone application; rather, I wanted it to be generated from some Javascript running within a browser. I could not easily find some functionality which did this, so I decided to roll my own.

The generator is here. It’s a simple JQuery based Javascript app embedded in HTML5 (ish)/CSS3 page. It’s been validated by the W3C HTML Validator, so it might work across browsers; I tested it on Chrome 6.0 on Mac OS X Leopard and it did what I wanted it to do.

One issue which I did not really think about before starting this small work was the Cross Site Scripting (XSS) issue – it’s not possible to request an arbitrary web page from Javascript that is downloaded from a specific server. This did impose limitations on what’s possible – essentially, I’ll have to add this page to any web app that I need to use it in.

The system comprises of a single page. At present, it’s housed in an AppEngine application, but as all the functionality is in a single web page, it can be just downloaded, copied on to any server and used to send requests to the server. (My AppEngine app has some extra backend functionality to make sure the client side is doing what is should be doing).

What functionalities does it provide? It can generate GETs and POSTs (my main motivation was to have a simple way to generate POSTs), it can emulate different user agents, parameters can be added to the request and the response is shown on the page.

The functionality is certainly quite limited – it’s easy to envisage lots of ways to extend it – but it was a reasonable test tool for some of the web app development I was doing.

I may extend it in the future – improve the UI, add more control over the HTTP generated, improve the error checking, provide a way to make it easy to update, load jquery via google libraries API, modify the ‘URI’ terminology as it’s inaccurate etc – but for now, that’s all he wrote.

Comments/suggestions/reasons why this is redundant(!) welcome.

The blogging students…

The four students I’m working with on final year projects have put up their blogs. They’re at:

• http://hoosheer.blogspot.com/
• http://eoinoconnor.wordpress.com/
• http://www.ahern.ie/
• http://saniya4yp.wordpress.com/

They’re good kids, so there should be some interesting stuff appearing on those blogs in the coming months.

 

PWOnView – a window into ProgrammableWeb

I wrote a small tool for looking at the ProgrammableWeb dataset – it’s here. This post provides some background on the work.

For the uninitiated, ProgrammableWeb is a repository which stores information on publicly available APIs and mashups of said APIs. The repository is rich, currently comprising of over 2000 APIs and over 5000 mashups and quite a lot of useful information pertaining to the APIs and the mashups.

Obviously, this is not an enormous data set, but even still, understanding what it looks like is non-trivial. I wanted to develop some simple means of understanding what it looks like to get some feel for what types of mashups people are creating. I asked the friendly guys at PW if they would give me an API key and they kindly obliged.

I had some ideas on what to do, but they changed somewhat as I got some experience working with the data. My initial objective was to look at the classifications that the PW guys have developed and see how mashups relate to classifications; in particular, to understand which combinations of classifications result in large amounts of mashups.

I got some basic animation up and running in which I extracted the 10 classifications with the most APIs, as I thought these the most important. Clicking on one of these classifications gives information on the APIs that fall into that classification, ordered in terms of the numbers of mashups that contain that API. So, for example, the first four APIs returned when the ‘Social’ classification is clicked are twitter, facebook, foursquare and LinkedIn. It’s then possible to click on each of these in turn to obtain a list of mashups which use those APIs, ordered by popularity.

(Note that the animation of the different classifications is not so important – originally, I had designs on making these moving circles that would move the selected classification into the centre and link to the other classifications, with the thickness of the link reflecting how many mashups comprise of APIs that fall into the two classifications).

Once I had developed this somewhat rudimentary visualization tool, I was able to navigate the PW data set a bit more easily. After playing with it a little, it became apparent that the dataset is somewhat skewed – there are over 2000 mashups which use the Google Maps API alone and youtube, flickr and twitter all come in around the 500 mark. Further, there are some classifications that have very few mashups.

This had implications for the work – in particular, I decided not to progress with enhancing the tool to reflect the linkages between the classifications as there was significant non-uniformity in it which is reasonably easy to detect.

There are still some interesting things which could be done with this and I may come back to revisit it at some future date. For now, it’s parked here.

Happy to have any comments/feedback on the tool in the handy comment box below!

The second(ish) Dublin GTUG meet

We had the second Dublin GTUG last night (<pedantry>although there seems to be a little bit of confusion regarding whether it is the second or the third – I’m going with the former, on the basis that the very first meet was before we agreed to proceed with a GTUG and hence was somehow a pre-GTUG meet</pedantry>).

We had a full agenda and a full house – I didn’t count, but I think there were about 30 people in attendance, which is very good by the standards of developer meetups in Dublin.

First up was Brian Brazil, an SRE at Google, who gave us some insight into the life of an SRE. He talked about some of the challenges faced by SREs in scaling up applications, understanding where bottlenecks lie and optimizing system performance. Achieving acceptable latency for service delivery is an important part of this job and requires understanding where delays arise throughout both the network external to Google and inside Google’s architecture – such is their dedication to delivering low latency services over wide geographies that sometimes even the speed of light becomes a problem! Brian did a particularly good job, as he was called in at 15 minutes notice due to a last minute change of plans as Nick Johnson unfortunately could not make it – respect!

Next up was Kevin Godden of Ridge Solutions talking about integration of Paypal and GAE. Kevin described how he had worked on a project in which it was required to use Paypal as a payment processor for a GAE hosted app. Kevin noted that GAE does not support HTTPS connections if you’re not using the appspot.com domain, ie if you’re running your GAE app from your-domain.com. This has implications for payments, naturally, and the only really workable solution for Paypal integration at present is to redirect the user to Paypal to take the payment (rather than taking the payment via the GAE app itself). Kevin talked about how Paypal’s redirect works and gave example Python code for integrating it with a GAE app. Link to Kevin’s slides.

After Kevin, there was a short, but impressive demo of an app built on GAE given by Ken Macleod of Aladdin Schools. Ken built Aladdin schools in a few months and the feature set that he put together was very impressive indeed. Everyone who had attended an Irish primary school got flashbacks when he showed the roll call functions on his app!

Finally, Jim O’Leary from Rococo Software talked about their experience in working with Bluetooth on mobile devices in general and Android in particular. Jim explained that BT on Android provides a limited set of the full range of BT functions, but this set has been well chosen and provides good coverage for most of the Bluetooth use cases. Working with BT on Android has not given rise to many problems in his experience, although he has one issue relating to making the device discoverable – Android imposes limitations on how long a device can be discoverable for.

The session in Google was rounded off with a Eoin giving out a couple of books for people to review and some swag for anyone who had earned it.

We then retired to the Schoolhouse for an engaging chat about all things G.

Overall, I was very satisfied with how it went – there was good engagement from everyone in the room, with lots of questions being thrown at the speakers and I know that I learnt a lot. Let’s hope we can keep this standard up as we go forward.

Thanks to everyone who stood up to to talk – great job by all – and thanks to Jean Joswig for being the Google contact who sorts out the room, pizzas and just generally making sure everything runs smoothly.

Looking forward to next month’s GTUG…

Protected: Academia and Open Source

This content is password protected. To view it please enter your password below:

Password: