Custom Symfony2 CLI output

October 13th, 2013 No comments

Just a quick little post for my future self.

I’ve recently been working on a little hobby project involving some intensive CLI stuff with Symfony2. I felt the output handler was lacking though, it was a small thing but I really wanted a prefix for each output with a time and a time difference between the last message. Simple stuff to see how long certain steps took in the process I was working with.

I did some googling and it was actually rather easy to add with symfony2. You simply have to add a custom ConsoleOutput which extends the normal one. This is what the one I made looks like.

<?php

namespace testPrj\ProcessingBundle\Component;
use Symfony\Component\Console\Output\ConsoleOutput;

use Symfony\Component\Console\Formatter\OutputFormatterInterface;
use Symfony\Component\Console\Output\ConsoleOutputInterface;

class ConsoleTimeStampOutput extends ConsoleOutput implements ConsoleOutputInterface
{
    protected $lastTime = 0;

    public function __construct($verbosity = self::VERBOSITY_NORMAL, $decorated = null, OutputFormatterInterface $formatter = null)
    {
        $this->lastTime = microtime(true);
        parent::__construct($verbosity = self::VERBOSITY_NORMAL, $decorated = null, $formatter = null);
    }

    protected function doWrite($message, $newline)
    {
        $message = $this->addTimeStamp($message);
        parent::doWrite($message, $newline);
    }

    protected function addTimeStamp($message)
    {
        $now = microtime(true);
        $diff = number_format($now - $this->lastTime, 5);
        $message = "[".date('H:i:s')."][{$diff}] "  . $message;

        $this->lastTime = microtime(true);
        return $message;
    }
}

Then in the CLI file app/console I added.

// Custom Output handler
use testPrj\ProcessingBundle\Component\ConsoleTimeStampOutput;

$output = new ConsoleTimeStampOutput();
...
$application->run($input, $output);

And that was that.

Now when I print output via $output->writeln() it comes out as

[20:09:30][0.00299] Pulling till 33778503  - 2013-10-12
[20:13:34][244.00970] [20:13:34] http://example.org :: Cache:False
[20:13:34][0.38880] 33649011 - 2013-10-05
[20:13:34][0.00005] Memory: 19.962341308594MB
[20:13:34][0.00003] fnd records:158
[20:13:34][0.00003] new records:42
[20:13:34][0.00003] sent records: 200
[20:13:35][0.28439] Done
Categories: PHP, software development Tags: , ,

the incredibly lazy guide to installing mod_pagespeed

November 12th, 2010 No comments

You hate reading? you want to try out mod_pagespeed? you run a ubuntu or other debian based server? Well then just follow the following steps.

  1. get the binary package based on your architecture. (to check which one run “uname -m”. If it says x86_64, they you have a 64bit server)
    • 64 bit.
      wget https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-beta_current_amd64.deb
    • 32 bit.
      wget https://dl-ssl.google.com/dl/linux/direct/mod-pagespeed-beta_current_i386.deb
  2. install the package (substitute am64.deb with i386.deb if you don’t have a 64bit version)
    sudo dpkg -i mod-pagespeed-beta_current_amd64.deb
  3. open up the following file with your favorite editor
    /etc/apache2/mods-available/pagespeed.conf
  4. add all the cool features you want, i currently run this. (line 47 in the file, but it doesn’t really matter)
    ModPagespeedEnableFilters collapse_whitespace,elide_attributes
    ModPagespeedEnableFilters combine_css,rewrite_css,move_css_to_head,inline_css
    ModPagespeedEnableFilters rewrite_javascript,inline_javascript
    ModPagespeedEnableFilters rewrite_images,insert_img_dimensions
    ModPagespeedEnableFilters extend_cache
    ModPagespeedEnableFilters remove_quotes,remove_comments
  5. restart apache
    sudo service apache2 restart
  6. done.

I haven’t fully looked into mod_pagespeed and all its filters and implications there of myself, but I always like following these kinds of lazy quick guides myself to start poking around instead of actually reading something for a change. So i figured I should just make one as well.

phing + dbdeploy website deployment

November 8th, 2010 1 comment

I recently had a project with xs2theworld to help create the mobile websites for intel asia. Because this project was quite important and I wanted to step up my game I created a proper deployment strategy. No more sweaty palms while running custom scripts, pressing svn up or switching symlinks. I wanted a fully automated deployment. A deployment I could test, run and always get the same result.

Because I’ve been hearing about phing and dbdeploy from dragonbe and harrieverveer I looked into them. They ended up being excellent tools to reach my goal.

Phing is a deployment tool in which you can create a deployment “script” made up out of a ant like syntax using xml.

<copy todir="${buildDir}" >
  <fileset dir="${projRoot}">
    <include name="**" />
  </fileset>
</copy>

DBdeploy is a tool that compares your patches to your database and creates a forward SQL patch and a backwards SQL patch, aggregating your SQL patches in the forward file and the undo statements in the backwards file.

In this blog post I will highlight some of the things I did.

I created an actual ‘build’ stage, where all the website elements were processed and copied into a separate build directory. The reason for this was two-fold. Firstly I wanted to be able to check the result of a ‘build’ without it being deployed, especially the SQL patches that dbdeploy generated. Secondly, I wanted to only copy those files that were needed for the site to run. So no .git directory, no sql patches directory, etc..
This has really been a great choice, because of the separate build stage i’ve at least had two instances in which i caught a problem before deployment. Saving me from the embarrassment of having to make quick fixes while the site was in offline mode.

I created separate phing property files for different environments. (production, staging, development) this combined with a simple wrapper script that called phing resulted in a very pleasant way of deploying by just issuing a command like “./deploy build development” or “./deploy rollout production” and the inevitable “./deploy rollback production”. Much better then “phing -Denvironment=staging build”.  Property files are basically ini files that contain key/value pairs that can be referenced from within your phing build file.
Then in phing you can say “<property file=”deploy/${environment}.properties” />” and it will read the property file. Please note that “${environment}” refers to a variable. which in my case was set when calling phing. (the -Denvironment=)

Dbdeploy is a piece of software that can read your SQL patch files, compare them to the database and create a single SQL file you can run to update your database. Unfortunately dbdeploy is very fussy about the separator you use between your patch and your undo patch. Yes, undo patch. At some point you want to rollback a deployment and at that time you really do not want to find out that you can’t because the new table structure breaks the old code.
It only takes very little time to create undo statements and you will avoid excruciating minutes of frantically applying changes manually when things break.
Also when creating undo statements be sure to set them in the reverse sequence of your normal sql patch statements.
e.g.

ALTER TABLE `myrecords`  ADD `rank` int NOT NULL;
RENAME TABLE `myrecords`  TO `myrecord`;

-- //@UNDO

RENAME TABLE `myrecord` TO `myrecords`;
ALTER TABLE `myrecords` DROP `rank`;

Also, that is how you should write the undo separator. Exactly like that. If you don’t dbdeploy will simply add the undo section to your deployment SQL file as well. Which is very much unwanted.
Also be sure to always, ALWAYS, ALWAYS! run both your forward SQL patch as well as your backwards SQL patch to make sure it works. Preferably not on production.

That’s about it I guess. There are many wonderful guides that will explain how to use phing and dbdeploy in detail, which is the reason I didn’t. I just wanted to pass along some things I used and thought worked nicely.
I would like to point people who want to read more about phing to the following blog posts that helped me heaps:
- Diving into Phing I http://groups.drupal.org/node/4363
- Diving into Phing II http://groups.drupal.org/node/5400
- Phing Build File http://sean.gravener.net/blog/web-design/phing-build-file/134/
- How To: Simple database migrations with Phing and DbDeploy (*this was a bit outdated*) http://www.davedevelopment.co.uk/2008/04/14/how-to-simple-database-migrations-with-phing-and-dbdeploy/

most important though.
http://phing.info
http://dbdeploy.com/

traveling elephpant

June 30th, 2010 5 comments

So a little while back ibuildings had this fun contest to build a path finding program that would solve a traveling salesman like problem. The constraints where pretty simple, you got a CSV file with latitude/longitude locations. You started at a certain location and you should end at a certain location. The application should then find the shortest route that touched all locations.

Now I’ll be the first to say that PHP is really not the language for that. A few years back I wrote a pathfinding tool for use with a game called EVE online in which I calculated certain trade routes based on data you could export from the game. After seeing PHP’s performance I switched to Python and more or less sliced the processing time in half if not more. Mainly because Python has specific array like types and PHP just has generic array’s, which with large data sets matters a lot. Perhaps also because my pathfinding-foo was still rather weak :)

However, back to now and the ibuildings challenge. The challenge would be rated on 3 criteria. Speed, lines of code and code complexity. Personally I could care less about the latter two and focused purely on speed. In the weeks that followed I had a great time comparing execution times with Remi and Andries. I think this was also key to keep diving into it and tweaking it until it was as fast as I could get it.

Sadly, my submission actually had a off-by-one bug in it which more or less instantly disqualified my entry. Yes, bit of a bummer, but such is life.

Now I had already decided to publish my code after the contest, however sadly I never really found the time to type this up. So a bit late but here is the code for my solution for the ibuildings elephpant challenge.

Below is the submitted code, and a link to download the code and original test data file so you can try it out for yourself.

download the code – Just unpack and run via php contest.php elephpant_landmarks.csv

-edit-

Ok scratch the code, I’m having some trouble with getting it to play nice. Just download the tar.gz and view the code in your favorite editor.


running ubuntu on a vaio BZ series laptop

January 18th, 2010 4 comments

I recently purchased a sony vaio VGN-BZ31VT. To be short, everything works for as far as I know and care.

Specs

CPU: Intel® Core™2 Duo-processor P8700 @ 2,53 Ghz
mem: DDR2 SDRAM (2 x 2 GB)
graphics: Mobile Intel® Graphics Media Accelerator 4500MHD

Wifi: intel wifi link 5100
audio: intel HD audio
ethernet: Intel 82567 Gigabit
bluetooth: ?  2.0 + EDR

what works

Well all the basics seem to work, the special function keys on the keyboard, the mouse pad, the screen, wifi and ethernet port.
But most importantly, suspend and hibernate also work. All of this out of the box, just install and go.

incidentally, this CPU also support intel VT, it is off by default, but you can easily enable it in the bios. If like me you use virtual machines a lot, it is rather nice to have. Haven’t done any real tests to see if it is faster, but at least it’s there.

Not tested

I haven’t tested bluetooth, don’t need it.

What doesn’t work

The laptop also has a fingerprint scanner, which with some tinkering can be used. It’s not so much a problem of hardware support it seems, but more that there isn’t a mainstream way of integrating fingerprints scanners with the security system in linux. The solution I read needs you to install some fingerprint scanning software and load that as a module in PAM. Too much work for too little gain for my taste, but if you really want it, then you can get it to work (probably).

conclusion

I wanted a no-nonsense development laptop with lots of memory and preferably virtualization support in the CPU, it should also work under linux with minimal fuss and suspend working was a must have.
Mission successful it seems.

Seeing as there is very little recent user experience info about this laptop out there at the moment, i figured i should write this little blog, if only to give people the peace of mind that you can safely buy this laptop for running linux.

Categories: hardware Tags: , ,

user settings cookie

January 17th, 2010 No comments

Sometimes in applications you will have certain user settings that you want to apply, even when the user is not logged in. Take for instance these examples:

  • “welcome back <name>” msg on return.
  • You have a portal type page where the user can control what content is shown where
  • You want to track where the user was when he last visited the site, perhaps to offer him the option to return to there.

I recently needed some functionality like that. So I’ve created a object that can help me with that.

I thought about it for a moment and created a singleton settings object for me to call upon to set and retrieve certain settings. Now I have to warn you that there is a small problem with singletons, if you use unit testing it can be difficult to control the behaviour of singletons over multiple tests. So be wary of this when you are running unit tests.

I also wrap all data in a separate array. This isn’t really necessary, but it makes handling the data a lot easier. If you wanted you could also add some sort of encryption to the cookie data so that users couldn’t easily tamper with it.

easy and simple transparency effect using GIF

October 3rd, 2009 1 comment

Transparency in HTML/CSS is largely an already solved problem, recent browsers all seem to handle PNG transparency pretty well and there are scripts that will make sure older browsers will handle them as well.

transparant gif exampleHowever, I wanted to make a post about a little technique I rarely see used which I think is quite genius in it’s simplicity.  Whenever you want to create a semi-transparent surface you create a gif file that contains a simple pattern of transparent and opaque pixels; as in the example on the right.  The white you see in the chequered image is of course transparent.

So let’s demonstrate how this effect actually looks.

transparency examples

Now as you can see the effect itself is very specific, and different backgrounds have different outcomes for the effect. Which might not fit every design. Another disadvantage is that it can only be used to show a 50% transparency effect. There might be pixel patterns that will give you a different distribution but I’ve never seen them.

The biggest advantage however is that you don’t need any fancy CSS or javascript or PNG, which in certain specific cases can be a big plus.  It’s more of a hack on your eyes/brain then on the browser :)

Categories: design Tags: , , , ,

Simple design rules for webdevelopers

September 4th, 2009 No comments

Designers. Can’t live with them, can’t live without them. More often then not, developers will have to work together with designers to create a website. Which most of the time means the designer will create a design and some HTML & CSS. Which the developer will then integrate and adapt to fit into his software to actually make the site work.

The problem however is; Developers normally didn’t go to design school and many of them have the artistic ability of your average garden rock. I’m certainly no exception to that. However, if you learn a few basic guidelines and rules, you can make the life of your designer buddy a lot happier by not screwing up his design.

So here are just a few general hints and tips to explain what is important when adding something to a existing design or when integrating it into the actual software of the site.

Aligning stuff

Stuff needs to be aligned, both horizontally and vertically. It must be aligned “visually” instead of accurately, which means that if you look at it, it should look aligned. Which could mean two pixels to the right or left of the point where two blocks where actually aligned.

however as rule of thumb you can pretty much aligned them accurately. The reason for aligning stuff is because it is visually pleasing for us to look at.

misaligned

aligned

In some few cases the designer might actually want to purposefully have a few elements be misaligned, so if your doubting, just look at his designs or simply ask him.

Whitespace

Where saying less is more. Whitespace, as the name suggests, is the empty space around objects. As with alignment, everything should have a little bit of white space and for the most part, the amount of whitespace should be the same.

Whitespace brings some calm into a design. A design will look far less crowded with ample use. Also, whitespace will often be used to emphasise certain elements within a design. This is where the amount of whitespace will differ. Headers for instance will often have more whitespace around them then paragraphs.

It is also always a good idea to make sure there is a lot of whitespace around important elements for your website. When there is a lot of whitespace around a element in comparison to other elements on a page, your average human will read that element first.

If you want to know a lot more about whitespace, read this article from a list apart that deals exclusively with whitespace.

colours

Colours are not only pretty, but also important. Often companies and brands will have very specific colours associated with them. Coca Cola red, UPS gold&brown, etc… Now designers could probably bore you for hours about colour theory and all that stuff. However what’s important for us developers to know, is to never introduce new colours. A designer will have chosen a small select group of colours for use in the design.

You will have one or two base colours, and a accent colour. For instance, for the design of this site, the base colours are light grey and white. and the accent colour is blue, and perhaps black.

What this means is that if I should add some new element, it should be one of those colours. Not purple simply because I like purple. Your best bet, depending on the size of the new element, would be to use the accent colour. The use of the accent colour is to basically add some spice to a design and use it to draw the attention of viewers.

When using colours, be precise. You might think, blue is blue is blue. But if your designer used #7c95e7 in his design, then be sure to use #7c95e7 as well. You could even ask your designer to write down the used colours for you in HEX.

ask your designer

Above are just a few general tips to watch for, but the best advice I can give is to simply ask your designer when in doubt or even just to verify. Because basically, It is like you created the software for the site and then some amateur takes it and start modifying it in small ways. I think most developers would at least like steer how those modifications are made, explain a bit what the thoughts where behind how it was written and such.

I have to say though, not all designers will like explaining it to you. But just remember, your not doing the above to please the designer, your doing the above to please the client, to make a better website. If that means having to bother your designer for 5 minutes, then so be it.

Why I think test driven development suck

August 29th, 2009 24 comments

TDD - All code is guilty until proven innocentWell I figured a provocative title like that would catch your attention, so please read on to find out I am not as stupid as you might think.

Unit tests are great, they reduce unknown bugs and arguably lead to better object design. The reasons for this are simple; Well thought out unit tests will force you to think about the input and output of any method your objects may have, forcing you to correctly decomposition the needed functionality within your object, because methods that have too much responsibility are a pain in the ass to write unit tests for. Simple one task methods however are easy to write unit tests for. Because your writing unit tests you are (hopefully) actively thinking about what can go wrong so that methods can actually fail.

So taking for granted that the above is true, why does test driven development suck?
Well the answer to that is a market one and really only applies to professional business. When a client comes to your business and asks for a offer for his project, you will quote him X, where X is roughly the amount of hours it takes to create the unit tests and then fill in the code + other stuff.

A development house that doesn’t use test driven development will quote that same client only for the time it takes to write the code + other stuff. We all know that after that he will spend a few weeks/months bugfixing, but that’s not what he quotes or tells the client.

Naturally the client will more often then not choose your competitor, unless you can positively convince him that the extra hours you spend will actually save him money in the long run. Which from the clients standpoint is a bit of an unproven claim if he is not technically inclined. So from a business standpoint, test driven development sucks as a selling tool.

Secondly, while this heavily depends on the morality of your business, those bugfix hours can be profitable. Depending on how smooth your talk is, you could very well be billing the client for those hours.

Thirdly, test driven development often suffers when deadlines need to be met. We all know that estimated hours is an imperfect science and often goes wrong, be it overconfident developers or overselling sales people, deadlines tend to be missed in our industry. So when deadlines are creeping, unit tests are often one of the first things to be dropped off schedule. Yes, we all know that shouldn’t happen, but it does. Once this has happened and no extra time is allocated to bring the tests up-to-date again (good luck getting that approved), the test driven development has failed and you’re just plainly developing again.

So for these reasons I think test driven development sucks. I might sometimes use it for private projects or open-source projects (for which I think it is absolutely brilliant), but certainly not for business ones. With the possible exception of in-house developed products, but I haven’t had much experience with that.

For anyone who now still thinks TDD is a sound development methodology, I would be very interested to hear you thoughts.

website text writing 101

August 26th, 2009 No comments

Now when people talk about websites and usability, a lot of stuff gets mentioned. Web 2.0, ajax, usability guidelines, designs etc.. etc.. What I however almost never hear is about the text. Sure people talk about content, user generated content, tagging, searching, etc..

But what about the little things? What about the buttons, the links, the subject lines of the e-mail someone receives? Those are the most read elements on your website, yet get the least amount of love.

So today I’m going to share some of my ideas on these texts, because I think they are important and should be treated as such. It all hit me about a year ago, at the DPC08, in the closing keynote by Terry Chay. He mentioned that when they added a smiley face to the subject line of some sort of mail they sent they saw a 20% increase in clickrate. That is nothing to scoff about. This got me thinking about how important these small texts are.

Over the year I’ve found a few key ideas about these texts that I think are important to keep in mind when creating them.

Positive Association
When a user comes on a site he or she will have a lot of actions to take. Design and content placement determines a large part of what a user will do. But what about texts? Now lets say our goal is to get as many users to register. We could have a “register” link. The link is placed prominent and the design helps the user focus on it so that we are almost certain that the user will read it. So what can we now do to actually let the user click the link?

Your average register link, contains basically the word “register”. Hmmm. Now I don’t know about you, but registering is annoying. I have to fill in all kinds of info about myself and give them my e-mail address, which I don’t actually want to do.

join linkedin

It all sounds a bit too much like a command. What we want to do is create a short text that reminds us of the fun things about having an account. “Join the community”, “share your pictures”, “create your account” All much better then “register”, the text should be inviting and friendly.

Leading the Userwould you link to comment?
When users visit sites we want to be able to guide them trough the site. We can effectively do this by emphasising what we want them to do via text. A good example of this is to be found on flickr. If you are not logged in, and you are viewing a photo, the site asks you “Would you like to comment?” instead of the often seen “To comment you have to be logged in”. If you think about it, the difference is miles apart. The flickr one makes me think, ‘do I want to comment?’ the other one doesn’t. I just think ‘oh ok’.

you must be logged in to leave a commentThe bit under both those texts is of course the same “sign up or login”. The difference however, is that after I’ve read the “Would you like to comment?” I actually either have decided I want to comment, or not. While with the other, I have no incentive at all to sign up or login. Or at least not if I wasn’t already planning to comment. Basically the flickr one reminds me that I can comment, and I even might want to comment.

Don’t Lie
It is however important not to lie. Every time a users gets confronted with something they did not expect, a percentage of them will quit. So for instance, in the positive association part I wrote “share your pictures” as text example for a button to sign up. But that’s of course not what the user expects. We promised the user they could share their pictures when they clicked that link.

So to take that as an example, we should make sure that after the user clicks that link we present the user with more text that reinforces that they will be able to share their pictures, just after filling in a few details.
That way we have given the user a clear incentive to register. This however often gets abused, I should make very clear that the page the user gets to see should not be a registration page, but a picture sharing page where you register. The actual sharing of pictures could of course be a step two of a wizard type structure.