Saturday, December 11, 2010

Updates to my bread and butter.

I just updated my YouTube library to incorporate a suggestion to allow direct uploading from your server to the YouTube servers. The new method is called "directUpload" and takes 4 parameters. The first is the path to the file to upload from your server. The second is the mime-type of the video (video/mp4, etc.). The third is the XML meta data that goes with this video, you can learn what this is here but it is essentially the same as what you would do for a form upload token. The final parameter is optional and defines the user whose account this belongs to, if you leave it blank it will go to the currently authenticated user.

Here is a sample call with the video to upload in the CI root directory:

$xml = '<?xml version="1.0"?><entry xmlns="http://www.w3.org/2005/Atom" xmlns:media="http://search.yahoo.com/mrss/" xmlns:yt="http://gdata.youtube.com/schemas/2007"><media:group><media:title type="plain">Test Video</media:title><media:description type="plain">This is a direct upload test</media:description><media:category scheme="http://gdata.youtube.com/schemas/2007/categories.cat">People</media:category><media:keywords>test, upload</media:keywords></media:group></entry>';

$this->youtube->directUpload("test.3gp", 'video/3gp', $xml);

You may wish to note that uploading a large file could take awhile and this library doesn't currently support resumable uploads so you just have to wait it out if things are taking awhile. As always you can turn on the DEBUG flag at the top and watch your error log for more info on what is going on.

While I was in there I took the liberty to fix a bug where only a response of 200 was deemed success by the library now any 200 level response is considered successful as it should be.

Here is the updated library: http://dl.dropbox.com/u/13081549/youtube.php

If you are new here consult my other posts for more information.

Monday, November 22, 2010

Neglected blog but not neglected code

Been awhile since I posted on here. Just been busy supporting my own sites as well as all the sites I have done for work. Any way I just wanted to point out that I have made an update to the youtube library I wrote for CodeIgniter. Apparently there was an issue with larger responses from YouTube not being handled properly. Basically for larger requests the data is broken into chunks with a specified size. The problem was that my read method was only reading the first chunk. Which wasn't a problem for me until I finally went over the one chunk threshold and was losing data. Anyway that is fixed and should now correctly get all chunks of data. (Though I could only test with 2 as that is my current threshold)

Also I added a new api method to that library which will allow you to post comments on a video or reply to a comment thread for a video. The method is called addComment and it accepts 2 parameters with an optional 3rd if you are replying. The first is the videoId which is the video to post the comment on, the next is the comment text, and the optional 3rd parameters is the commentId if you are responding.  

I also updated the wiki post on the codeigniter site here
And placed the latest version in my dropbox share here

One thing to note I think I left the code in debug mode, which shouldn't do anything really, it will just write out a bunch of stuff to your php error log. You can turn it off by setting the DEBUG flag to false.

[EDIT] Looks like there was another bug. If a youtube response had newlines in it, maybe in a description field or something, then we would only read to the new line and stop. I fixed this as well the updated version is on the drop box share. Also updates are on the wiki.

Tuesday, October 19, 2010

Support your developers

For a long time now I have used a number of free and/or open source applications. In fact I prefer them generally to paid applications for many reasons but namely because they are free. Lately though I have felt that the great folks who contribute these excellent software packages need and should see the support that they deserve. So I have begun contributing small donations to many of the products I use on a regular basis. I figure for something like Cyberduck which I use daily there is absolutely no reason to ignore their requests for donations. This program is indispensable in my day to day.

Please don't construe this as me requesting any kinds of donations from anyone I personally don't feel I have written anything worth any monetary value. But I do ask that you support the creators of the free applications that you use regularly, and make a small donation to them.

Wednesday, October 13, 2010

Twitter OAuth CodeIgniter Library, That was easy.

After spending months scratching my head and wondering if OAuth even works at all. I was able to crank out a library that will do OAuth for twitter in about 2 hrs. Yeah you are right I probably should have been actually doing my job but this is way more fun. It also gave me the opportunity to revisit my google OAuth library and fix the issues it had with HMAC signing.

Basically HMAC-SHA1 signing was impossible in the old google library, so I fixed that and it should still work the same if you are using RSA so don't worry about updating. If you want HMAC though you will need to jump through a few more hoops (nothing too serious) mainly involving keeping and using a token_secret value. I outline everything in the CI wiki if you are interested in how to use it and how it works.

Anyway back to the library I just finished. This library works pretty much the same exact way as the google library the only difference is twitter only lets you use HMAC signing. This means that you need to maintain a token secret value. Not a huge deal but one more thing to keep track of. If you want a more technical look at the implementation check out the CI wiki post I made here

Otherwise if you would rather just download the file you can get it from my dropbox account:
http://dl.dropbox.com/u/13081549/twitter_oauth.php

You also need the oauth_helper I wrote you can get it from dropbox here:
http://dl.dropbox.com/u/13081549/oauth_helper.php
If you want to see what it does then visit the wiki page here

Using this library is very similar to using the google oauth library here is an example:
function request_token()
{
  $params = array('key'=>'CONSUMER_KEY', 'secret'=>'CONSUMER_SECRET');
  $this->load->library('twitter_oauth', $params);
  $response = $this->twitter_oauth->get_request_token(site_url("site/access_token"));
  $this->session->set_userdata('token_secret', $response['token_secret']);
  redirect($response['redirect']);
}

function access_token()
{
  $params = array('key'=>'CONSUMER_KEY', 'secret'=>'CONSUMER_SECRET');
  $this->load->library('twitter_oauth', $params);
  $response = $this->twitter_oauth->get_access_token(false, $this->session->userdata('token_secret'));
  $this->_store_in_db($response);
}
The response array returned from the get_access_token call contains the access token, the access token secret, the twitter user id, and twitter screen name. You can do what you want with the last two values but you need to store the access token and token secret as these are how you authenticate with twitter.

[EDIT]
It was pointed out to me that once you have an OAuth access token it isn't exactly clear what you do with it. The OAuth portion of your API requests goes in the Authorization section of the header. The oauth_helper provides a nice function that will create the desired Authorization section.

$access = array('oauth_token'=>urlencode($this->session->userdata('oauth_token')), 'oauth_token_secret'=>urlencode($this->session->userdata('oauth_token_secret')));
$extra = array('status'=>rawurlencode('Tweeting from my API'));
$authhdr = get_auth_header('http://api.twitter.com/1/statuses/update.json', 'CONSUMER KEY', 'CONSUMER SECRET', array_merge($access, $extra), 'POST', OAUTH_ALGORITHMS::HMAC_SHA1);

The rest of the header is up to you. Make sure you include the values you are posting in the get_auth_header extra field, and that it is rawurlencoded.

[EDIT 2]
I also ran into an issue where I did not specify a callback when I registered the application with twitter. If you don't specify a callback, there is a bug with the registration process that will force your application to be set as a desktop application. If you are getting an index out of bounds exception when you make your first request make sure you have a callback set for your application on twitter.

Tuesday, October 12, 2010

CodeIgniter and the YouTube API. This is a triumph


 Please note that the code for this has been moved to GitHub Here


I just want to make a note here, 'huge success'.

I finished work on my youtube API library for CodeIgniter. This will let you connect to youtube and make requests without having to load any Zend GData libraries. It is written exclusively for CodeIgniter.

To use the library you must first pass in some parameters. The apikey is the apikey provided to you once you register with youtube to use the API (I put mine in a config file). Next you want to include the oauth consumer key that you setup when you setup oauth with google. The secret is the consumer secret google provides you if you are using HMAC-SHA1 signing or the file path to your .pem file if you are using RSA-SHA1 signing (again I put these in a config file for easy access/changes). You must then specify the signing algorithm you use to make your oauth signatures. Finally you have to specify the access token and token secret if your user has already authorized your site to connect to youtube on their behalf.

If any of this seems over your head please consult the oauth docs on youtube for further clarification. You can find those here. It should also be noted that this library only does OAuth authentication it does not support AuthSub feel free to add this if you would like though.

This library does require the oauth_helper I wrote for CodeIgniter I will link to the file at the bottom of the page but if you wish to further understand what it does visit the CodeIgniter Wiki here. Also I wrote a library to handle all of the OAuth interactions that is also available on the Wiki here. (I will provide a link to this at the bottom of this post too.)

$params['apikey'] = $this->config->item('youtube_api_key');
$params['oauth']['key'] = $this->config->item('google_consumer_key');
$params['oauth']['secret'] = $this->config->item('google_consumer_secret');
$params['oauth']['algorithm'] = 'RSA-SHA1';
$params['oauth']['access_token'] = array('oauth_token'=>urlencode($token));
$this->load->library('youtube', $params);

That's it you now have a youtube API instance and can make calls. The following methods are available please consult the documentation for usage etc.

Here is a list of methods that are available right out of the box, no authentication needed.

$this->youtube->getVideoEntry($videoId);
$this->youtube->getRelatedVideoFeed($videoId, $params = array());
$this->youtube->getVideoResponseFeed($videoId, $params = array());
$this->youtube->getKeywordVideoFeed(array $keywords, $params = array());
$this->youtube->getVideoCommentFeed($videoId, $params = array());
$this->youtube->getTopRatedVideoFeed($params = array());
$this->youtube->getMostViewedVideoFeed($params = array());
$this->youtube->getRecentlyFeaturedVideoFeed($params = array());
$this->youtube->getWatchOnMobileVideoFeed($params = array());

The list of methods below requires an authenticated user before they can be used.

$this->youtube->getUserPlaylistFeed($user = 'default', $params = array());
$this->youtube->getPlaylistFeed($playlistid, $params = array());
$this->youtube->getSubscriptionFeed($user = 'default', $params = array());
$this->youtube->getContactFeed($user = 'default', $params = array());
$this->youtube->getUserUploads($user = 'default', $params = array());
$this->youtube->getUserFavorites($user = 'default', $params = array());
$this->youtube->getUserProfile($user = 'default');
$this->youtube->getUserActivity($user = 'default');
$this->youtube->getInboxFeedForCurrentUser();

Finally these methods all require some type of data to be specified whether that be XML meta data, a comment or a boolean value.

$this->youtube->directUpload($path, $contenttype, $metadata, $user = 'default');
$this->youtube->getFormUploadToken($metadata);
$this->youtube->addComment($videoId, $comment, $commentId = false);
$this->youtube->addVideoResponse($videoId, $responseId);
$this->youtube->addNumericRating($videoId, $rating);
$this->youtube->addLikeDislike($videoId, $like);

All methods where the user is defined as 'default', or are getting data from the current user, and the form upload token method, all require oauth authentication or they wont work. The methods that let you define the user may work without authentication but will only show data that the specified user has made public. The form upload token method requires xml meta data to be passed in as well. You can see what this should look like here. All API request return XML and it is your job to parse it and get what you want from it.  

I should note that I haven't tested all of the methods defined in this library so I cannot say with 100% certainty that they all work, but from the ones I have tested I can make a pretty good assumption that they do. Also I used RSA-SHA1 signing for my OAuth requests I just couldn't get HMAC to work properly I found an issue with my signing method for HMAC and fixed it but I cannot guarantee that it works properly either. If you have an issue post it in the comments and give me your HTTP header information for both your request and the response. There is a DEBUG constant flag you can set that will output this information to your error logs. I am also going to post a more technical brief of this on the CodeIgniter wiki which I will link to here.

[EDIT] There seems to be some confusion about how all this stuff fits together so let me clarify the steps. Please Please Please read all the documentation before leaving a comment.
  1. Download the linked files and put them in the correct CI directories. You need all 3 of them. Two go in the application/libraries folder and one goes in the application/helpers folder.
  2. Read the documentation on OAuth. Having a basic understanding of how it works is key. Make sure you read the oauth_helper and google_oauth wiki pages. Both contain vital information about generating the necessary keys and how to use those files. Read the oauth_helper wiki first. It has links on generating some keys and how to use them. Don't worry too much about understanding how oauth_helper works it is more for internal purposes but you do need to checkout the links in its wiki. Read the google_oauth wiki next as it provides more working examples of what you need to do to get oauth up and running and get the necessary access token.
  3. Make your controller to get a request and access token. Samples are provided on the google_oauth wiki page. If you have read the documentation up to this point then it shouldn't be too difficult to figure out what to do. Just use the google_oauth wiki controller examples for reference.
  4. Test your controller methods out and make sure you are getting an access token and there are no hiccups in the 3 step process. You can get an access token as many times as you need so don't worry if you messed something up and need to get a new one.
  5. Read the youtube api wiki page and check out the official youtube api documentation which can be found here
  6. Make your controller to access the youtube api. You will pass in the access token you got from the previous steps as well as an API key you get from google. Make sure you read up on all of the methods the api provides. It is by no means a complete API library there are a number of missing features which may get added in the future but my hope is that you can figure out how to add the things you need by reading the official youtube api documentation and looking at how I implemented things in the api library. Really what it boils down to is HTTP header manipulation.
  7. Finally put everything together. The process should be pretty seamless with the user only having to leave your site for a split second to approve the request token. Also note that you can do this via a local server so even if you setup everything to your sites domain if you are running a dev instance locally everything should still work.
If you have read everything on this blog post and all the subsequent pages it links to and you still have questions please let me know. Judging from some of the questions I have had all of the info is in the blog post and the linked pages I think the problem is: there isn't a clear order to it all. For that I apologize. Follow the steps outlined and it should make things a bit easier.

Monday, October 11, 2010

Komodo Edit 6 is out?

It has just come to my attention that Komodo Edit 6 is out. I am kind of surprised as I attempted to update from 5 yesterday and no updates were found according to the built in update method.

I think I will hold off for the time being though. I have a number of toolboxes defined and am right in the middle of a project so I would rather not risk it. Soon though, I need to finish reading the change list. I hope things are a bit more stable on Mac as it seems that Komodo Edit doesn't like being open for long stretches (3-4 days) and then have a number of tabs closed. It would pretty consistently crash when those stars aligned.

SVN Support in Komodo Edit

One of the features that makes a great IDE is version control support. Some really nice features like the ability to see what lines have changed in a file and revert a single line are built in to many IDEs such as NetBeans. However due to the issues I outlined in an earlier post NetBeans just won't work for me on the Mac. So I reverted to the tried and true Komodo Edit IDE.

Subversion support does not exist out of the box in Komodo Edit, and once you get it working it is much more basic than the grandiose features of other IDEs but it works and it is certainly better than nothing. You can follow the instructions found here or you can just download the KPZ file there. One thing I discovered was missing from this toolbox though is the ability to delete from subversion. I added another tool that gives this ability, it relies on you just deleting the added file then clicking the subversion delete button. Sorry about the ordering it appears that Komodo Edit only lets you set toolbox items alphabetically.

Your order of operations prior to check in now becomes (Note 1 and 2 are interchangeable) :
1) Click the Subversion add button. This will pickup any new files in your project.
2) Click the Subversion delete button. This will remove any files deleted from your project.
3) Click the Subversion check in button. To finally commit your project.

You can read the blog I linked to for info on how to checkout a project and how to update.

These files are now hosted in GitHub along with a similar tool set for Git Version Control.
Here is the link to the GitHub Repository

If you want to read more about the Git tool set check out my blog post about them.

Thursday, October 7, 2010

Hmm WordPress

According to my boss I am supposed to be learning how to work with wordpress. I have never been very good at independent learning I am more the kind of person who needs a project with a deadline to really dig into the guts of an architecture. I will begrudgingly install it locally and fiddle around a bit but I doubt I will go beyond that until I get assigned a project.

Tuesday, October 5, 2010

Finished with phase one of GamerGameOn

I have finished the first phase of development on Gamer Game On and now I have to attempt to drum up support and get some users on the site. This usually proves more difficult than the actual development. My plan is to use all the social media I can. I just hope that the site is appealing enough that people will want to be a part of it. 

Hooray for CodeIgniter Code Completion

Now you might have already seen it, but there are methods of getting code completion for native CodeIgniter objects and methods. You can look here for an example that works in NetBeans. Well I have just made the discovery that this will also work in Komodo Edit, which is currently my editor of choice. All you need to do is create a file at the root of your project (I named mine autocomplete.php). Then you put all of the @property lines in it that were defined in the post I just linked. Restart Komodo Edit, and voila you got yourself some code completion.

Though it is not automatic per se if you create a custom library or model you can add those to your auto complete file and they will show up exactly the same. 

While not perfect it is much better than nothing. Also with Komodo Edit's project templates you can make a CodeIgniter base project that already includes an auto complete file. The world is your oyster now!

Here is a link to the autocomplete.php file I use in Komodo Edit. Just drop this into your project root (the same place as the .kpf file) restart Komodo Edit and you should have native CodeIgniter auto completion. If you have custom libraries add them where specified.

http://dl.dropbox.com/u/13081549/autocomplete.php

Monday, October 4, 2010

Social Network wasn't bad...

I saw the movie 'The Social Network' it wasn't bad, I especially appreciated the scenes at the beginning where they talk about apache configurations and different techniques to acquire what you want from a web server. Also, while some people might miss it, I appreciated the complete obsession the character had for his creation, from hardly sleeping, to constantly checking stats. It truly embodied what I feel as a developer when I am in the midst of a project. Nothing else matters only what you have done so far and what is left to do. 

That being said I think it paints Mark Zuckerburg as more of a programming genius than he really is. I don't personally know Mark, but given Facebooks humble beginnings I look at what they have done and often think "I could have easily done that" and there are some cases where I did do something before Facebook, but given the small scope it went virtually unrecognized.

It should be noted that though he may not be a genius from a technical standpoint, Mark definitely showed, in the early days, that he could make a product that people wanted. There in lies his genius, or luck maybe. Facebook in the early days was fun to use, easy to understand, and exclusive. All of these things have absolutely nothing to do with the underlying code, so painting Zuckerburg as some kind of coding guru because he made these sites I feel is a misrepresentation. He could have very easily been an idea man, had someone else do the code work, and still achieved the same result.

I think if you are a programmer you will appreciate this movie more than the average person, because you will be able to recognize that this is a case of the American dream come to life. You will see that if you can figure out what people want, then you can be the next Mark Zuckerburg. As a programmer that gives me great hope, though it is probably a one in a million chance.

Thursday, September 30, 2010

A pox on NetBeans for OSX!!

I am always on the look out for a great IDE and for the price (free) NetBeans is truly one of the greats, for any web dev. It has great code completion and even lets you define custom code completion for things like CodeIgniter. Where the loading stucture tends to confuse standard code completion methods.

As I said it truly is a great IDE unless of course you are on a Mac. Unfortunately do to the way they package their executable in OSX, NetBeans cannot open a file from finder. Meaning if you open finder and double click, or select open with NetBeans on a php file or some other code file NetBeans will start if it hasn't already but will never open the file in question.

Now 70% of the time this isn't an issue using the tree view project structure on the left to open your files in your project is pretty basic and straight forward, but if you ever need to open a file out side of that you are forced to use File -> Open menu options on NetBeans, something I think hardly anyone does and something that is extremely annoying.

After having a bug ticket on this open since 2008 myself and a few others finally got enough attention drawn to the matter to get it escalated to have someone look at it. So a month or so goes by and finally the results are in: Closed, Wont Fix.

Now I do understand why they wont fix the issue. Who in their right mind would completely rewrite some of the base code of their project just to handle an annoyance in a single OS? I know I wouldn't, and I can't blame them for that either. It just sucks that this issue is annoying enough to me that I am now looking for a new IDE in OSX that can handle the "magic" of CodeIgniter. Currently I use Komodo Edit which does an alright job, but lacks many of the great features of a true IDE, and no CI code completion, or debug support.

My requirements for a good IDE are, debug support, subversion support, CI code completion, a good find/replace implementation, and works in OSX obviously. I have tried Eclipse but its bulk tends to leave a bad taste in my mouth before the program even finishes loading. That may be my only option though (sad face)

You Got CodeIgniter In My ExpressionEngine

Since the release of ExpressionEngine 2.0 I have been writing plugins for it like crazy. There have been a few frustrations here and there. Also some WTF moments when I look at the EE code (Have you seen how they do custom fields in the database?). Overall though I have had little trouble bending EE to my will.

Some neat plugins I have written which due to my current employment might be sold instead of given away despite my own personal wishes.

A simple input plugin that handles both get and post input values. (There are tons of these but I am particularly proud of how I was able to leverage CI for an extremely clean code set.)

A field injection plugin which takes on EE field and puts it into another at a specified position. 

A simple email plugin that sends an email out. (There are tons of these)

A module to populate a channel via CSV. This one probably took me the longest and is the most complex, it also brought to the surface a code bug in EE that I made a bug ticket for you can read more about it here: http://expressionengine.com/bug_tracker/bug/13549/

I also wrote a plugin that is an extremely simple shopping cart implementation it is unique in that it doesn't have any kind of check out methods or anything of that nature it is just a cart (as the plugin's name implies). This was quite a challenge because for some reason EE does not grant access to a users session. They actually override the CI implementation which despite many complaints still remains true. I was able to get around this by copying CI's session library and renaming it, then using it as a custom library. I documented all the steps involved here: http://expressionengine.com/archived_forums/viewthread/138599/P18/

After working out that session stuff I made a plugin around that as well.

All in all having CI to fall back on (in most cases) makes plugin writing for EE a dream. It is amazingly easy to get EE to do whatever you want, and you already have a powerful CMS on your hands.

Wednesday, September 29, 2010

CodeIgniter, OAuth, Youtube, and Me

The site I am currently working on GamerGameOn requires authentication with youtube to pull videos to show. Needless to say this has been quite a task. OAuth documentation while good tends to be quite technical, and in some cases, out of date but getting it to work isn't impossible and most of the issues I have encountered have ended up being my own fault. Here is my youtube library. I wrote this instead of using the Zend GData implementation because I didn't want to include a number of other libraries and files that I wouldn't be using. The link to the article is here: http://codeigniter.com/wiki/OAuth_for_Google/  This is just the OAuth request and access library the actual youtube library needs some cleaning up but it all works pretty well. Read the doc in there for a very detailed look at OAuth for google and CodeIgniter.

As I say in the wiki post this is supposedly a general Google OAuth library according to docs all you need to do is change the scope constant to have it work with another google service. I never tested this so I don't know if it's true but in theory it should work.

Here are dropbox links to the files if you would rather download them.

http://dl.dropbox.com/u/13081549/oauth_helper.php
http://dl.dropbox.com/u/13081549/google_oauth.php

CodeIgniter

CodeIgniter is currently my framework of choice. I love it's small size and useful libraries I try to contribute as much as I can on the CodeIgniter wiki. With handy helpers and libraries like: OAuth Helper and Language Library Extension I have a ton of libraries besides those which I will probably share on this site instead of the Wiki which is a bit bloated and difficult to navigate.

OAuth Helper is meant to aid in signing OAuth requests which can be pretty tricky. All you do is provide your consumer key, consumer secret (either a string or a path to a .pem cert file), then also an array of additional parameters (so if you have an oauth_token and oauth_token_secret)

The Language library extension provides some functionality which seems to be missing from the stock CodeIgniter language library, which is the ability to add a parameter to a language string. It uses standard sprintf notation in the language line and takes passed in parameters of either a single string or an array of strings.

Finally on the Blog Wagon

Well I have finally made a blog. The main purpose of this is to have all of my code in a single place more for my own use than anything else. If someone else can glean any new knowledge from the stuff I have learned then I consider that a bonus. I don't know how often I will post but you can bet that when I do it will knock your socks off.