Advanced Flow Control With NodeJS and Q Promises

9 Jun

I’ve been playing with node for a while now and have been through all the popular flow control libraries (q, step, async) while working on various projects but for my latest piece of work I needed something a bit extra. What I was finding was that while “doing these stages in series” and “performing these steps in parallel” was all very well and good, it was simply too trivial for what I needed to achieve. I also found that in all of the use cases it was known upfront exactly what requests were to be fired and in what order, maybe it’s just me but this is rarely the case.

I needed something more dynamic, I wanted to be able to basically customise the flow depending on my route and do things like “fire these x requests together, then when they are all done, do this request then after that do these x more request”. So I came up with a short piece of code to do not only that, but also take a json configuration to control it.

https://github.com/kevinhodges/advancedFlowControlWithQ

You can clone the repo and hit:


node .

Here’s what’s going on.

The variable myTasks is a structured JSON object. Inside it, each “stage” (stage1 and stage2) gets executed in series while its value array, are calls to other services which get executed in parallel.

Request data is the json that gets passed in at the top of your stack, this will typically be request data if you are running an API.

We then get down to the two core functions.

runAllTheseStagesInSeries()

Notice the first thing we do here is create a scope variable and attach the request data to it. As the stack executes and the stages get processed, scope will be appended with the result returned in the promise by each “helper” so we have great visibility on whats going on at every level of the process.

We then create an empty promise to make sure our stages are executed in order. As we loop each stage, the promise chain is appended to and the contents of each stage (all those requests we want to execute in parallel) are passed into the second function getAllTheThings().

Lastly we use a Q.allSettled to make sure we wait for all operations before proceeding to the next stage.

getAllTheThings()

Looping all the requests passed for the current stage we execute the appropriate helper defined by the request. In the first instance we are calling the package createFullname and in the second ageInTenYears. The key parameter tells getAllTheThings() what key to use when appending to the scope variable.

Again at this level, only once all promises are settled do we return the outer promise and continue to the next stage.

A quick note on helpers, you may notice the helpers directory with 2 files in. These are simple stub functions that take a deferred and perform a certain function with the .do() name. These could be http requests to 3rd parties, reading files of disk or whatever your heart desires, essentially though once it has finished executing it must either resolve or reject the deferred.

Wrapping up, the beauty of this solution is that as each stage gets executed we know exactly what has happened in previous stages and can act upon that info. You can make this more useful but allowing stages to take extra configuration telling it which part of the scope to perform its package on.

 

Mobile first CSS for decrepit browsers – Let’s sort it once and for all

22 May

So, if like me you have been dealing with responsive websites in any way, shape, or form, you have 100% heard the term “Mobile first”. Mobile first is brilliant, and we all should be thinking about it when developing our sites. There are some brilliant ideas floating around at the moment on the subject, so have a read…

The one problem I have found when trying to code with this practice is trying to support browsers that just don’t want to play media query! So, this article is going to go over a few possible solutions for handling media queries.

Right, so we have an example CSS file. The best way to visualise this is for me to throw together a dummy CSS file:

.hiddenOnWideScreen {  display: block;}

@media all and (min-width: 750px) {
    .hiddenOnWideScreen {    display: none;  }
}

So, this is spot on. Think about our little screens first, then hide our random block once we get to our wider devices. The downside of this though, is that all the browsers out there that don’t support media queries (IE7, IE8 and the rest are here) will stretch our designs and techniques such as small screen grids. So let’s talk solutions!

Option 1:
We could use conditional stylesheets, most projects have them anyway. Set them up like this:

<!—[if lt IE 9]> <link rel=“stylesheet” type=“text/css” href=“site.ie.css” /> <![endif]-->

So, we’ve got our stylesheet. Then inside of this we do something like this:

.hiddenOnWideScreen {  display: none!important;}

Sorted! The project isn’t delayed…let’s go home! Oh wait, we have 6000 lines of CSS and it’s going to get awfully messy trying to overwrite them all line by line, style by style.
Option 2:
Let’s just imagine a set width for our browsers to receive, write a programme to run through our CSS file, and strip out the media queries that don’t fit our set size. It will be just like the olden days!
Stop writing that programme! We can just borrow grunt-stripmq bytilomitra (what a guy!). It’s a grunt plugin that does exactly what we want to do. Woohoo!
Add this into our Gruntfile, preferably as part of grunt watch so this will run automatically, then we can put something like the code below into our final production code. Then IE7 & 8 will see websites exactly how they understand and the world will be a better place…

<!—[if lt IE 9]><link rel=“stylesheet” href=“site.ie.css”><![endif]-->
<!—[if gte IE 9]><link rel=“stylesheet” href=“site.css”><![endif]-->

Oh wait, as we know with projects, this is the part of it where the client walks in with her laptop running Firefox 2.0 and is furious that the website looks so horrible
Option 3:
So, this probably wouldn’t happen, and I’m sure if somebody looked at your flashy new site in Firefox 2.0 the lack of media queries would be the least of your worries, but still, we have that brilliant no media query file.
Why don’t we serve it up to the people with browsers that don’t support them? Simple enough, we just have to use modernizer to check for media query support, then swap out our CSS file depending on that support.
I propose something like this:

<link rel=“stylesheet” id=“noMediaCss” href=“/path/to/css/site.css” type=“text/css” media=“all”>

Drop a span into the page with a data attribute attached.

<span id=“noMediaCss” data-nomediacss=“/path/to/css/site.nomedia.css”></span>

Then use modernizer’s mq function to detect if we have media queries enabled or not. If we don’t, perform the old switcheroo!

if ( Modernizr.mq( ‘only all’ ) === false ) {  $(‘#mainCss’).attr(‘href’, $(‘#noMediaCss’).data(‘nomediacss’) );}

So all browsers that don’t support media queries now have our non responsive site showing. I think we can tick the progressive enhancement box. These are just some ideas though, if you have other techniques please share them with me. I would be really interested to see what everybody else is thinking on this topic.

AWS Enterprise Summit

1 May

Tuesday Rob and I attended London’s AWS Enterprise Summit, it was a day designed for their customers to gain more of an understanding about what products and features are available with the amazon suite, speak to some of the official partners about extras that could be achieved to manage their account in a more effective way and have a general chat to the Amazon team / other Developers.

The day was kicked off by Stephen Schmidt, Vice President, Amazon Web Services giving us an overview on the kinds of products Amazon provide. A lot of it felt like he was preaching to the converted, being a day of enterprise accounts, but everyone has a different route to getting on to the cloud and developing their AWS stack so some may have found it worthwhile. For me some of the statistics were stunning, every 16 seconds they update a version of software. They released 280 features last year and 56 features since February this year! Of course these could be small scale releases so out of context don’t necessarily add up but over the last year we have seen constant improvement to our service so it all boosts the package. He informed us we all gain from the customers with higher needs as every improvement that they make is provided to everyone. In his words  ‘The rising tide floats all boats higher’

ImageImage

He also mentioned about the recent price cuts to their packages, apparently there have been 42 price drops since 2006, and that the more services Amazon provide for you the better these savings are. This was reiterated to us in the case studies time after time. He suggested that those who hadn’t should speak with their account executives to assess if their set up was still running in the most cost effective way for their company.

Image

AWS hosted talks from UK based enterprise companies with real life use case examples two of our favorites were from: News UK and Shop Direct

Chris Birch, IT Director, News UK

His case study featured around The Times On Sunday and their online subscription. They have 150,000 subscribers who want the latest news between 8am – 10am but how during other periods their servers would require see less load. Moving to the cloud gave them the capabilities to scale their infrastructure to meet this need, and made the concept of an online subscription based business more than just a hardware intensive idea. Their story wasn’t an easy one, they hit the 17 transactions a second limit every week for months requiring them to be on call continuously but with their advisor got through the processes. They now have more than 450 million transactions a month and with the recent price cuts, cut £4000 off their bill every month.

Image

Andy Wolfe, CIO, Shop Direct

Shop Direct are the company behind big name brands like Littlewoods, Isme and Very. These days 80% of their traffic is online with 40% of that mobile giving them 770k unique visits per day across their brands. The cloud gave them the capabilities to react quickly to the customers especially during the ‘golden quarter’ of Christmas. He told us about the importance to scale and their love of data; including their UXlab – which I hear is something to brag about! He explained how using Cloudreach; An Amazon Partner; helped their journey to the cloud, they consider themselves more a traditional company and found the support network reassuring.

During lunch there was opportunity to talk with approximately 38 partners with expertise in a range of AWS products offering varying levels of service depending on the customers needs. I found that a lot of the stands had a high level of what I considered ‘sales’ staff rather than ‘technical’ staff, which was a shame as after spending a few minutes explaining my AWS stack and the kind of challenges we faced with it I was often told I should speak to another guy at the stand or they’ll get back to me. Followed by the scanning of my ticket and the start of the junk mail that is already arriving in my inbox from stands who couldn’t provide me any assistance. Still most of them told me ‘It seems like you’re doing pretty well.. erm.. yer..’ if that’s any consolation to other developers…

I found that no one really wanted to discuss continuous integration with me or my colleague even though we both tweeted in the hope someone would like to discuss it with us. The guys at the Chef stand gave it a fair go and where very knowledgable about their product even offering future training possibilities. ( BTW our Dev team loved your tee-shirt swag ). Many admitted it was a topic that comes up but not something they could really help with.

Image

The other thing that became apparent is there is a HUGE gap in the market for a node.js solution to diagnose and fix application performance problems in real time. There were a few partners there that covered this area but none of them could comment on how or when they would be improving their node support, but are aware it was lacking. My company have hugely embraced node.js SO we really hope this message got back to your development teams as one of you will make a killing in the marketplace.

We found out that both our Account Manager and Solution Architect where at the event, having never met them before this was a huge WIN for the day. They and all of the AWS staff where hugely helpful throughout the day and a credit to the event being a success.

Another big highlight of the day came from a chance meeting or rather falling into Just Eats twitter social experiment. After a day of sales there is nothing like the prospect of a good developer chat and a beer. Coming from different industry backgrounds based in different tech languages, their good humor, similar use of APIs and AWS knowledge made for several hours of conversation that continued into the evenings drinks. I believe these guys have a lot to do with the London AWS meetup, knowing that, I have added the date to my calendar and am sure it will be worth the drive up to London.

All in all, it was an enjoyable day. The food was good, the free bar was fantastic, the customer talks were impressive, and network opportunities where vast. They even managed to get the day to run on time even with the tube strikes! Just perhaps a slight tweak to the level of technical staff on partner stands for next year.

FEDs take on Future of Web Design April 2014

30 Apr
On a spring day in London, Marcus and I entered the Brewery to attend the Future of Web Design 2014.  FOWD has a history of being held in high regard so we went with some trepidation and excitement.

KICKOFF
The initial kickoff by Paul Boag was welcoming and immersive (but lacked the pizazz & energy of Reasons.to).  Disappointingly we were told that there had been visa issues with Sara Soueidan, one of the speakers I’d hoped to see talking about The future of web layout with CSS shapes.  Also Westley Knight who was planning a about Digital Agency (R)evolution was unable to attend.
Image
There were 2 full days of tracks intersected with plenty of coffee, food and chat.  Marcus and I split the tracks down the middle for maximum coverage across both days.

TRACKS
The tracks I attended were a good length (ie. not to long), and great content.  The standout speakers for me were Joshua Marshall, Todd Motto, Jason Paremental & Peter Gasston.  If you do not follow them on twitter already I’d recommend it.  These speakers and their content were engaging and I learned or affirmed a lot from it.

THE BEST 
Joshua Marshall - The accessible web of the future
When first joined GDS (Government Development Service) he was given 6 weeks to prototype gov.uk.  This site has gone on to replace 24 main government websites, 110 local gov sites, released 110 domains, and recycled the majority of existing content.
The spec was to design a site that was accessible for both civil servants and general public to use.
How do you make this work for everyone?
Started with Design Principles, between the designers, developers and content teams strip the current back to make it as simple as possible.  They didn’t design with disabled in mind, they designed with simplicity in mind to make it easy for everyone, regardless of ability.
Then set up Service Standard, what the site looks like
Finally the Service Design Manual – take all the knowledge and turn it into a manual and make it public. Sharing is caring.
Where does accessibility fit in?
Introduce it early to avoid a snowball – it is not optional
gov.uk uses the bbc.co.uk accessibility guidelines – again, on the public domain so utilise!
Accessibility is a quality issue – this is a no-brainer for professionals in the web
You will always get a better product through empathy, considering users first and removing ambiguity.
By incorporating WAI-ARIA they have helped their disabled users with screen readers, datepickers, and buttons.
Finally web design is 95% typeography; type setting, glyphs, kerns and all that will make or break your content.
Now?
gov.uk has gone on to win numerous awards for design that have previously been held by Apple.  No mean feat.
Image
- – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - -
Todd Motto – Demystifying JavaScript
Todd was a front end developer who only knew jQuery and when going for jobs was balked at for not knowing actual JS so he went and immersed himself and is now one of 100 Google Developer Experts.  Currently he’s got the luxury of developing purely for Chrome, no IE!
Javascript is the language of the moment, the future is JS & HTML5.  Forgetting jQuery, performance is a bit key of JS.
The big misconceptions – jQuery:
- Helps me write less code
- always have me covered
- the only way to develop front end code
- a must for cross browser code; ajax for instance
- a must for complex selections
The big misconceptions- JavaScript
- it’s difficult to learn
- it isn’t cross-browser without writing loads of code
- isn’t necessary because jQuery exists
What have these misconceptions done to JS?
- created a fear of JS
- reliance on jQuery
- new developers bypass and go straight to jQuery
- claim it’s hard to find JS resources, replies jQuery biased
- obscured what JS really is
- “Just use jQuery”
The Power of jQuery
- It came at the right time, “it was a ship that everyone jumped on to get us to where we needed to be, it’s time to jump off”
- Ease for cross browser
- easy and consistent api
Why do I need to learn JS
- we’re not going to need huge libraries in the future
- JS is faster than jQuery – outperforms around 84% faster
- Reliance on jQuery isn’t sustainable
- Total control and understanding over “Everything” that you write
- jQuery is heavily DOM focused
- Remember HTML5 & JS are the future, not jQuery
- – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - – - -
Jason Parmental – Type, responsively
Our canvas is constantly shifting. How we deal with type in that context changes has to be with something more interesting than Arial. First spoke at Drupal con. 2011. Used the phrase responsive typography. How you can hold a device and how the visual interpretation will change.
Type that’s actually responsive is a ways off.  Responsive Typography is more than just webfonts
responsive must be:
- performant 
webfont hides the text until font loads, just load
– you have 5 seconds until the user leaves – be careful about load time
– Don’t waste users time
– respect the web, respect your users (content first) get the content on the screen.
– Font loader classes – Fout is your fault. .wf-inactive / .wf-inactive fallback
– tune fallback – specific css to avoid reflow
– test your design with fallback fonts only
– adjust font-size, line-height, letter-spacing wto avoid jumpiness
– Design for readability & beauty – only load the type you need from webfonts. Don’t forget font is BOLD NORMAL ITALIC, gothic is typeface
- progressive
– tuning fallbacks ensure it looks good, fast & addresses fastest-growing segment.
– use both css & js for broadest support (css in noscript tag)
– chose your stance: assume js w/fallback or tune for no js or @font-face & build up
– Asia India Africa are huge markets which mostly run Opera Mini which doesn’t support font-face. including iconfonts – 350 Million users never see your site as you intend
- proportional 
We constantly let our users down, thinking about how much we can cram n the screen instead of the message we want them to read.
– much has been written about RWD
– responsive images
– emerging ui patterns
– Small canvas requires subtle proportions
– what works in print works in print
– bringhurst matters, but scale must adapt
– line length is important but use device norms for size
– think relative proportion, not specific screensize
– test = font quality is improving
- Polished
– type is your voice, speak eloquently
– real quotation marks are brilliant
– browsers are lazy letter-spacers
– Open type is awesome, see what the kerfuffle is all about.
– Use libraries like Typogrify & Smartypants (module in Drupal)
– Automatically replace quotes, &’s, more.
– never leave an orphan
– We mainly work with systems and the systems need to handle this because content is not intelligent to deal with this.  – — Protect the design and the integrity of the content being added.
– Loose letters look sloppy
– browsers aren’t so great at it
– try letter spacing -1px in your headers, makes them belong rather then just disconnected letters.
Opentype feature fest
– real kearning tables
– ligatures fractions and swashes – oh my – design gold
– every design element must have a purpose
– even if the purpose is simply greater beauty. There are elements that should delight the user.
SO what is it. Yes, it’s a thing!
– adapting to screen size, network speed and decide capabilities
– its’ about designing for what’s next, these core feature will fit with things moving forwards. “by being more backwards compatible we are going to be more future friendly”

MEETS
There are some plus points from being a smoker as I tend to find networking a darn sight easier.  There were a real diverse mix of people who we chatted with including:
- 3 of the Danish development team from Trustpilot (they booed Feefo) (https://twitter.com/KristianThrane)
- 2 of the Essex based developers from nettuts/envato
- Todd, lead developer from motors.co.uk

SORE POINT
I think the biggest disappointment was with Harry Roberts talk on the Monday. He was ill, and from his sickbed had recorded his talk about CSS Frameworks.  I went along to the Main Stage not realising this and was met with an empty stage and a video of his slides with his voiceover, which I found really hard to engage with.  I wasn’t alone and in the 45 minutes I stayed I would think over half of the room cleared.  He was better and took over Saras slot on the tuesday but as I had already been at his first talk, albeit subpar on delivery, I opted to attend Pauls talk with Marcus.

Talking in public

14 Mar

Recently, I had the pleasure to be invited to talk at Code Harbour about “The Command Line”.

Code Harbour is usually based at The Workshop in Folkestone, a really cool new building featured quite heavily in the media for it’s indoor slide!

(If you’re in Kent, I’d highly recommend attending the next meetup, it’s a good mix of tech types with lively discussions and Pizza!)

I was a little nervous accepting the invitation to speak as I don’t consider myself an “expert”, for this reason, I did state this in one of my slides, mainly to cover myself in case things went wrong, but interestingly, it didn’t really dawn on me until I started speaking that it doesn’t matter.

Luckily, I was in good company. I’d attended a few Code Harbour meetups in the past and I knew most of the guys so it wasn’t too traumatic.

I’ve often believed that to talk in front of anybody, you had to be an “expert” or at least have some public speaking experience. For some reason, I’d always assumed that those people I’d seen at conferences in person or YouTube coverage of conferences were at the top of their respective fields.

Turns out, to talk in public, you just need to step up & well… talk in public!

“When you are an expert, it’s hard to explain what you know. It’s hard to explain your methodology etc. This is a shortcoming.”
Angelina Fabbro: JavaScript Masterclass

Preparing for my talk, I found I was learning MORE about the command line, about things I use EVERY DAY than ever before. I’d actually improved on the things I do on a daily basis away from the place where I use them most!

This got me to thinking… if I want to learn a new subject… could I learn a little, then simply do a talk about it? How risky would this be?

Obviously I’m not suggesting you go out & sign up to the biggest conference you can find just so you can learn a subject, that’s just crazy… I just believe that if you are struggling to grasp a concept, or just want to give your learning a boost… do a talk, even if it’s among friends / family / work mates.

Talking in public will force you to re-frame the subject in such a way that your brain will get a cognitive “kick” and you will reap the benefits.

Good luck!

To defer or not to defer

18 Feb

In a recent exercise to help improve page load speeds, we went through some of our sites, adding the defer attribute to some of our script tags.

<script src="localFile.js" defer></script>

It’s worth noting that you should not add the defer attribute to any file or library, that you depend on in any inline scripts. For example

<script src="//ajax.googleapis.com/ajax/libs/jquery/1.11.0/jquery.min.js" defer></script>
<script>
$( 'document' ).ready( function() {
    //Do some javascript here. This is a simple example
    alert( 'The DOM is now Ready' );
} );
</script>

will fail, reporting $ is not defined. This is easily fixed by removing the defer attribute on the jQuery script tag.

After adding the defer attribute across some of our pages, we started to notice some random behaviour.

We do have some scripts that depends on previously loaded scripts. This is why we chose defer and not use the async attribute, based on the theory that defer should load the scripts on the order in which they are found in the document.

When defer was first introduced a lot of browsers implementation was erratic (according to https://hacks.mozilla.org/2009/06/defer/) , but Firefox’s implementation was correct. Our testing showed that Firefox’s current implementation (as of Firefox 27.01) appears to load the deferred scripts in a random order, at least 1 of 10 times the page is loaded, but Chrome and Safari seam to work as per the specification.

Our solution is simple. Make sure any scripts which are depended on later are not deferred, until this issue is resolved.

Google Analytics Callbacks

2 Jan

If your using classic analytics from Google (ga.js), you might not know that it does actually provide a callback  within the track event function.

Use the code below to set your callback;

_gaq.push(['_set','hitCallback',yourFunction]);

This callback will stay valid for any further pushes, but can be cleared with;

_gaq.push(['_set', 'hitCallback', null]);

Hope this helps

Follow

Get every new post delivered to your Inbox.