SUNZ preso’s have been popular
A few years ago when I first left SAS and started up OptimalBI, I took on the role of chairperson for the SAS New Zealand User group.
As part of that role I started uploading the excellent presentations from each years SUNZ conference to slideshare.
I had forgotten all about it until I received an email today from Slideshare with some stats on how many times the slides had been viewed.
51,000 times, amazing!
Here are the stats (counts 
they sent me.
|
|||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||||
Interesting the majority of big hitters are related to SAS Enterprise Guide. But then again as it must be SAS's most popular product I suppose its not that surprising.
Histograms are not history
When I taught high school statistics [a life time ago!] there were a number of key principles that we’d attempt to drum into the students when it came to presenting their findings:
- Nothing should be given more focus than anything else (so no colour or anything else that would cause any part to stand out)
- Reporting should be transparent (give as much detail as possible to ensure any reader can be satisfied with the analysis, inferences and conclusions)
- Present the data truthfully – ensure the graphs do not mislead or distort by
- swapping axes around
- squishing up one axes to exaggerate or hide aspects of the data
- using 3D graphics (its actually very difficult to read 3D graphs)
- using exploding pies (or worst still exploding 3D pies)
Fast forward (ahem) a few years and the seemingly bland graphics (sigh) of yester-year do not seem to fit with today’s desire for colour, shades and multidimensional images – ‘sound bites’ or ‘pop’. Add in the current trend for ‘information graphics’ and the days of the humble two dimensional black and white histogram look numbered!
Are the old ways doomed?
Maybe not…
At the end of the day it comes down to how well you know your data and how thorough the analysis is. At the bare minimum you need to ensure you have a sense of the shape/distribution, the variability, the total counts across key groups, the relativity of changes, proportion of missing values, and what sort of outliers you have.
Once you have these you will have a sense what is key in the data (this is the story you will need to tell). To help with transparency run the outputs of your exploration/analysis by another analyst – this is very useful as once you’re in the data it’s hard to tell if your outputs give the full story or if you are filling in the gaps. Do they come to the same conclusion? If not, why not? Did they need additional information or did they notice something you hadn’t.
Ensure that you stick to telling the story of the data, not what you (or the board) want it to be - no artistic license can be invoked (e.g. Argo and its loose interpretations of NZs involvement!).
Do not underestimate the power of a simple, uncluttered graph to give your presentation POP.
- Use descriptive but succinct titles that actually reflect what’s in the graph.
- Ensure that you use the right chart for the data:
- bar charts for categorical data,
- if you must use a pie chart – use it only if you have around four to seven categories and never to compare,
- time series graphs for data with a time component.
- Dump the 3D graphs; they’re too hard to read.
- Use consistent fonts, styles and colours.
- They should be self-explanatory. Ask a non-analytic colleague to read through your charts, if their brain implodes they’re not self-explanatory and your presentation will not pop.
Traditionally highlighting key elements of data has been frowned upon. Busy managers however require explicit information, often in the form of ‘sound bites’ so it sticks. You could try for example:
- judicious sorting - highest to lowest rather than alphabetically
- using a darker shade of colour to have the data stand out (just make sure you’re within the brand guidelines)
- putting a box around a particular row in a table.
Take great care not to overdo it.
By providing supporting information in an appendix [e.g. relativity, variability, spread, shape, counts] there is no reason why key aspects of the data cannot be brought to the forefront. This also has the additional advantage of providing ‘detail focussed’ people or other analysts what they need for their understanding.
So yes you can still ensure transparency and truth while providing pop and sound-bites.
What is a good culture? (apart from blue cheese of course)
Thinking back to all the places I have worked before I started OptimalBI, a lot of my memories are rose tinted (enhanced) or dark grey (clouded) by the culture that I perceived was endemic at the company at that time.
Often the smaller team I was part of exhibited a different culture to the rest of the organisation (so times for the better sometimes not). And I can also remember times when a new member of the team joined and the culture changed (again sometimes for the better sometimes not).
I have been slightly fixated on trying to guide OptimalBI to have a culture that represented a company I have always wanted to work for.
When Vic joined OptimalBI a few years ago, we decided that if we weren't having fun we were doing something wrong. Not to say we wanted to play all day and do no work, being unemployed with no money is not fun, so OptimalBI still needed to deliver value to our customers and be paid for that value we delivered.
When we did one of our first planning session we were lucky to have stumbled across Valves Employee Handbook. The style and sentiments of that handbook embodied a lot of what kin of company I wanted to work at (hell I spent so many hours finding a desk with wheels we could buy so we could move them around the office when we felt like it like Valve).
Lately I have found a couple of other companies who's style and culture are similar to what we are striving for at OptimalBI.
One of those is Hubspot and their CultureDeck.
In researching company manifesto's and culture decks, I found a lot of articles with a positive opinion of Hubspots culture deck (and the Netflix one that they based the idea on). Plus a few that were a little less complimentary.
Here are links to the articles in case you feel like having a read.
- http://www.linkedin.com/today/post/article/20130320171133-658789-culture-code-building-a-company-you-love?
- http://onstartups.com/tabid/3339/bid/97200/Does-HubSpot-Walk-The-Talk-On-Its-Culture-Code.aspx
- http://www.slideshare.net/reed2001/culture-1798664
- http://37signals.com/svn/posts/3450-when-culture-turns-into-policy
- http://blog.prettylittlestatemachine.com/blog/2013/02/20/what-your-culture-really-says/
AskSAM – How to INTNX using SAS
AskSAM is an exclusive club for Optimal BI customers only; the meeting of gurus (and wannabe gurus). A place where no question is stupid and every answer is valued. A community of users sharing their knowledge to help others reach their guru potential.
If you would like to know more about AskSAM please email asksam@optimalbi.com.
Appreciating SAS in a Microsoft World – Date Manipulation
Recently I've been involved with building a data warehouse environment for an organisation that has decided that Microsoft SQL was the answer. I’m continuously finding myself comparing the world of T-SQL with that of SAS.
In many respects, I think, the tools and technologies provided by both Microsoft SQL Server 2012 and SAS 9.x are a bit of a jumbled mess – VB, C#, Java, SCL, Data step, SQL, SSIS, the list goes on – how is the poor developer and administrator meant to cope! But that’s not what I want to talk about…let’s get back to basics. The tools and technologies are only as good as the functionality that they provided and nine times out of ten this is the realised through code to manipulate data and I think MS SQL is still playing catch up in the data warehousing and analysis world.
My latest exercise is migrating Oracle P/SQL to Microsoft T-SQL to replicate some of the old reporting that was produced prior to the data warehouse implementation (don’t get me started on why this is so bad!) and generally it’s a bit of a find and replace job. The one standout thing that I see is the lengths developers have to go to to manipulate dates in both SQL languages.
For example, what I would consider a fairly common occurrence, the previous weekday day. Today is Tuesday, I want yesterday’s date - Monday; or more complicated, today is Monday and I want yesterday’s date – last Friday. In T-SQL this ended up as:
DATEADD(d, CASE WHEN DATENAME(dw, GETDATE()) = ‘Monday’ THEN -3 ELSE -1 END, GETDATE())
How bad is that to look at and computationally it can’t be good.
SAS on the other hand provides the enormously powerful and useful INTNX function which “increments a date, time, or datetime value by a given time interval, and returns a date, time, or datetime value”.
For example, I want to get the date five days from now. Using INTNX I can do this:
INTNX(‘DAY’, TODAY(), 5)
“Wow!” you say….how’s that any better than DATEADD? Well, we have barely scratched the surface.
Now I want to replicate the code above. In SAS this can be done like this:
INTNX(‘WEEKDAY71W’,TODAY() , -1)
The first parameter defines the interval that we want to work with, the second the start date and the third the shift. In this case the interval ‘WEEKDAYxW’ allows us to only work across weekdays where x defines the days to skip as weekends (here we are using Saturday and Sunday).
Now you have to admit that is a bit better.
SAS provides many intervals to work with – DAY, WEEK, MONTH, QUARTER, YEAR, etc. but the real power comes in when we these combined with multipler and shift-index values. The multipler lets as move over several periods in one hope, while the shift-index aligns to partial periods.
For example, ‘YEAR.6’ aligns us with the 6 month of the year, while ‘YEAR2.’ allows us to move across 2 years at a time.
INTNX(‘YEAR.8’, ’12MAR2012’d, 3) = ‘1AUG2014’d
INTNX(‘YEAR2.’, ‘12MAR2012’d, 3) = ‘1JAN2018’d
INTNX(‘YEAR2.4, ‘12MARCH2012’d, 3) = ‘1APR2016’d
INTNX provides an optional alignment parameter. This allows you to align the resulting date within the period you have shifted your date to. The default is the beginning of the period as you can see in the second example above. You can align your result to either the beginning, middle or end of the period or recently added, the option to align to the same as the start-from date.
INTNX(‘MONTH’, ’12MAR2012’d, 3, ‘B’) = ‘1JUN2012’d
INTNX(‘MONTH’, ’12MAR2012’d, 3, ‘M’) = ‘15JUN2012’d
INTNX(‘MONTH’, ’12MAR2012’d, 3, ‘E’) = ‘30JUN2012’d
INTNX(‘MONTH’, ’12MAR2012’d, 3, ‘S’) = ‘12JUN2012’d
Finally I leave you with a common example from both worlds to calculate the end of the previous financial year:
MS SQL:
CASE WHEN MONTH(GETDATE()) BETWEEN 7 AND 12 THEN DATEFROMPARTS(YEAR(GETDATE()), 6, 30)
ELSE DATEFROMPARTS(YEAR(DATEADD(YEAR,GETDATE(),-1), 6, 30) END
SAS:
INTNX(‘YEAR.7’, TODAY(), -1, ‘E’)
BI Platform in 3 months – Lesson #1 – Change your business owners engagement frequency
We have just finished a 3 month sprint working with a customer to deliver a BI Platform from woe to go in 3 elapsed months. This is the first time I have been involved in delivering something from scratch to production in such a short amount of time.
Yes we have done lots of small developments in that period and even done some semi-agile (id say it was iterative) developments where we went from inception to production in 3 months on a platform that was already production and stable.
But in this case we were talking about starting from nada and going live with data and some reports in 3 months.
It has been a wild ride and we have had some successes and some failures. We sure as nuts learnt a lot for when we do it again next time (and im happy that we will).
So im am going to post a series of short blogs on what I learnt, what I would do the same next time and what I wouldn't (as I find time and in no particular order!)
Lesson #1 - Change your business owners engagement frequency
So the first lesson for me was the needed to change the engagement timeframe with the business owners.
Typically you might have monthly steering committee meetings and weekly team meetings. But with a 3 month window this only allows for 3 steering committee meetings and by the time you get to them its to late.
Now one of the things about steering committees is they shouldnt be a route for escalation of issues or decisions on how things are done. They should be there solely to make sure the project is on time, on budget, delivering the scope that has been agreed and managing any risks acceptably. It's a health check process not a management/decision process. Of course we all know that's not how it normally goes.
So my suggestion is drop the steering committee meetings to fortnightly. Ideally these health checks will happen in line with the fortnightly development sprints you are doing, so they can get transparency of what is being delivered on time and what isn't.
Interestingly I think you will see your steering committee makeup change slightly as a result of the call on key people's time to attend fortnightly meetings.
The second suggestion is to have direct access to the key business owner on a daily basis for escalation of any issues. We all know every project gets a myriad of roadblock's that often escalate into "magic roundabouts". So you need access to the key business owner so they can ask, beg, plead, yell, demand, shout and threaten whenever you need them to remove these roadblocks. Remember a week's delay in a dependency will kill your project delivery timeframe.
Using Stored Processes with SAS Visual Analytics
There's been a lot of talk lately about SAS Visual Analytics. This is how typical report looks in SAS Visual Analytics 6.1:
From the above report it looks like each report object (Pie, Bar and Drop down) sits within its respective reporting window, but how about if you wish to embed the output of a SAS Stored Process in your report layout?
Lets try that by embedding a Stored Process in one of the existing SAS VA Reports.
Have a look at the following SAS VA Report named “Sample Report” which is a Bar chart created from sample data.
Now lets insert the Stored Process into this SAS VA Report so that Stored Process output sits alongside the Bar Chart Report Object.
To insert the Stored Process, Navigate to “Insert->Other->Stored Process”
Select Stored Process – “Sample: Multiple Output Formats” and click “Open”
Now a trick for young players if your using Google Chrome as your browser:
OOPS!!!!! Yes this is how the output of Stored Process looks in SAS VA in Chrome. Even though we can see clearly that it has a sufficient allocated reporting window to comfortably accommodate the output of the Stored Process output, it is rendered out of that window and it looks messy, not only that but it also disturbs the viewing of existing Bar Chart Report (overlap on the bar chart).
You won’t face this alignment issue in Firefox or IE Browser.
Properties Tab ->When you mark checkbox “Show log in output” (right pane), then the output shows the log as well.
Now when you mark checkbox “Show Metadata View” (right pane) then you will be able to see the STP metadata information.
If you want to change the output style then click on 
“Edit Prompts” icon (right pane)
Popup will open where you can select the Output format and ODS Style.
(In this example: Select Output Format: Plain HTML and ODS Style: Seaside) and click OK
Below view is for above Edit Prompts selections
So hopefully you are now able to quickly add output from a SAS Stored Process into your SAS Visual Analytics report!
AskSAM – How to INDEX using SAS
AskSAM is an exclusive club for Optimal BI customers only; the meeting of gurus (and wannabe gurus). A place where no question is stupid and every answer is valued. A community of users sharing their knowledge to help others reach their guru potential.
If you would like to know more about AskSAM please email asksam@optimalbi.com.
Some Innovations are big, some are small, some make the boat go faster
When looking at what value your company provides to its customers it can often be classified into the 3 E's
- Efficiency
You provide something faster or cheaper than the customer doing it themselves. - Experience
You have done something a number of times before and its better value or lower risk for a customer to get you to do it, than it is for them to learn to do it themselves. - Expertise
The customer is attempting something that hasn't been done before (or no experience of its exists in the local market) but your Experience in similar areas means its lower risk if you do it compared to them.
When looking at Innovative ideas it's often a good approach to see which classification the Innovation is likely to deliver value under:
- Changing a process to achieve efficiency gains
- Packaging up a solution to deliver lower risk
- Inventing something completely new that is a game changer
In the area of credit card payments we are seeing some Innovation occurring within certain retailers. You may not have noticed but at McDonalds if you pay by credit card and the purchase amount is under $30, you no longer need to enter a pin number.
A similar capability is now available at Countdown if you have a OneCard Visa. Any purchase under $80 with this card no longer requires a pin number. And in fact they go one step further and provide tap-to-go that means you no longer have to insert your card and select credit card, you just tap the machine and your away.
This Innovation is definitely aimed at the Efficiency E.
If you think about the time-saving achieved by this capability, you are probably going to save all of a minute in the process. Not much of an efficiency for you.
But if you take an example of 60 people going through the checkout at McDonalds or at Countdown, then we are talking about a one hour saving. This increased efficiency in the checkout process will result in increased throughput, and in turn will result in shorter queues.
I don't know about you but queue size is definitely one of the sub-conscious measures I know my brain uses to determine my level of happiness when going to a fast food outlet or a supermarket.
As an aside I am amazed that fast food outlets don't seem to use any sort of optimisation process to work out the level of staffing they need at peak times. I was at Burger King the other day during the lunchtime rush, they had two people on counter and one manager assisting them (no idea how many people out back). Both queues had over 10 people in line, as the poor counter people struggled to take orders and then fulfil them. Even without a statistically significant optimisation or forecasting model, let alone a social media based sentiment analysis, you could tell it was't working or giving customers a happy ending.
The other thing I noticed was there was no culture of achievement being displayed. The team were up against the odds and they were no doubt going to have to deal with grumpy customers, but they were just wombling around filling orders. If they had provided a sense of they were working as a team to try and "knock the bstdr off" then I think customers would have cit them a bit more slack.
Anyway next time you toying with an Innovative idea, have a go trying to work out which E it falls under.
AskSAM Advice – How do I change database schemas in a SAS/Access libname?
The Problem: Our database is setup to re-direct SAS connections to a database we don't have read/write access to. Is there a way you can change the schema name so that the library is assigned to the correct one?
Solution: dbconinit libname option.
Example:
For DB2
libname edwtest DB2 dbconinit="SET SCHEMA = SAS" datasrc=******* user=******** password=**********;
The dbconinit option passes any text after the '=' sign to the database you're using. Note that the commands in the text string are specific to the database you're connecting to. The above example changes a DB2 session to use the schema called 'SAS'. But there are other comands you can use.
This can also be used in SQL pass-thru.
Further information on the option and applicable database engines is available here