fredag 4. mars 2011

CIPA part 4: Local solutions: Gaining control

The public IT policy in Norway is one of purchase. That is, it is up to software development companies to create solutions that fulfill Norway-specific requirements, which the public administration in turn buys - at an ever increasing cost.

Consultants in Public Administration: Part 1 - Part 2 - Part 3 - Part 4 - Part 5

In many ways, one can say that a small group of software companies in Norway increasingly "own" the public sector as a result of this policy. And consultants are having a field day. I have often wondered what the rationale is and observed the following points:
  1. Large systems are required, and it doesn't make sense that all municipalities hire their own programmers to develop their own systems. Let someone else reinvent the wheel.
  2. When using open source software, there is noone to call if something goes wrong.
  3. Anything built in-house is at the mercy of the developer - what if he/she quits or gets run over? Nobody else will be able to take over. (Interestingly, this is also considered to be true by many even if the local developer makes his work open source.)
  4. Spending money on the private sector makes money go around and does, eventually, return as taxes. Spending money specifically on the IT industry thus strengthens the IT industry in the country.
There is certainly merit to all of these points. Yet, the local IT department in most municipalities still tend to be undermanned and spending a lot of time on extinguishing fires. Because large systems do crash, even if they are provided by a large vendor, even a few good men can not know everything. And what SHOULD be obvious, yet most don't fully realize: the more that happens outside the control of the IT department, the less control the IT department has.

What is going on around here?

When I started in my current position, most of the operations were outsourced to a consulting company that also had the contract for equipment purchaces - an obvious conflict of interest. Documentation was scarce. Sure, there were the passwords, there was a "kind of network map" that went into great details - albeit some of the most meaningfull details were missing, and the map had not been properly updated for four years.

Only a few days into the job, I would get a phone call asking how a certain case was going. I had no idea, I didn't know the case existed. I had to check with the consultant, who was there only two days a week. My little sheet of notes turned into a bunch of yellow postits, which turned into a hierarchic digital notebook, which eventually was impossible to keep track of. And documentation was still scarce!

Tools to regain control

The overlap time was spent putting together as much documentation as possible. These tools worked for me, though they should be evaluated as any tool be evaluated before using:

Network documentation consisted of drawings of cables with tiny annotations indicating VLAN and port numbers. This was an excellent outdated physical map, but did little to easily demonstrate how the VLANs were routed. A created new maps using FreeMind - one for the physical connectivity and one for VLAN routing. One of the reasons for choosing FreeMind, is that the files produced are actually XML-files, which can be interpreted and put in a mySQL database - and even reproduced on the basis of a mySQL database.

Software documentation went into a local MediaWiki site edited by all IT staff - hereafter known as the "ICT Wiki". Exceptions were made for passwords and other attachments that don't make sense to put on a wiki. Any piece of software we found out was part of our operations would go into the wiki as soon as we discovered their existance.

As far as support went, I had one guy for all the schools, and 90% of everything else was dealt with by the consultant. With 700+ employees, the lack of a support ticket system was screaming at me. I installed eTicket just to get on top of things.

Cleaning up

Next, I told every department to call IT operations first, not the consultant. And preferrably, send email directly to the ticket system. This way, we were able to gain a better understanding of what the issues were, do more in-house, build documentation, make a more unified design and use the consulting company only when needed.

The contract was eventually cancelled, as we no longer needed them for the minimum amount required to fulfill the contract.

A clean-up job is something that takes a really long time. The bigger the systems and the larger the network, the longer you will have small surprises pop up at you. In this 700+ employee, 12 location organization, surprises from "the consulting era" still pop up three years after. We are not done cleaning everything up, but we're slowly getting there.

Internal strategies

Software tools are not enough to get on top of things, though. There is also work flow. As IT professionals, we tend to strive for continual betterment, which in Lean-terms is called Kaizen. We couldn't turn the entire organization lean over night, but we started to turn the IT department lean by focusing on the time consuming tasks:
  • New employees: previously a two hour cost for the IT department, now 50 seconds.
  • Password reset: previously a two minute cost for the IT department several times a day, particularly in august and january - now a self service.
  • Moving empolyee between departments: previously 5 minutes of research and 2 minutes of doing - now a 10 second service. This will eventually become a self service for the department leaders.
Every task we find ourselves repeating over and over again are reviewed for automation. One must, however, carefully follow the procedures and remember to document what has been done in every ticket. In order to keep focus on current tasks, we have turned to Kanban. This is currently being tried out in different fashions, and I will be writing about my own experienced in this field.

Consultants in Public Administration: Part 1 - Part 2 - Part 3 - Part 4 - Part 5

1 kommentar:

  1. Godt skrive.

    Eg jobbar sjølv som innleigd konsulent for det offentlege, og kan bekrefte at mykje av det du seier er rett.

    I ei ideell verd, så burde det offentlege ha teke betre styringa over prosjekta sjølve, og berre brukt konsulentar som ei midlertidig løysing for å gjere utfyllande arbeid på prosjekta.

    Det som imidlertid skjer, er at det offentlege ikkje har gode nok leiarar og systemarkitektar blant sine ansatte. Så, når dei hyrer inn konsulentar, så skjer det veldig ofte at konsulentane "veit betre", og at dei dermed endar opp med å definere prosjekta, i tillegg til å faktisk implementere dei.

    Når ein får dette, i kombinasjon med konsulentane sine "vikarierande" motiv, så blir det rot og røyr.

    Problemet med motiva til konsulentane er som fylgjer:

    1) Dei har ei interesse i å tene penger, så dei vil kunne finne på å definere opp prosjekt berre for å generere inntekter for seg sjølv.

    2) Dei veit at dei på eit tidspunkt kjem til å forlate skuta. Dette gjer at dei tar snarvegar som dei ikkje ville ha teke dersom det var dei sjølve som skulle forvalte og drive systema vidare. Dette skaper rot og kostnadar.

    3) Dei har ei interesse i å nytte "cutting edge" teknologi, for å vidareutdanne seg sjølv medan dei utfører prosjektet. Dette gjer at dei kan finne på å bruke feil teknologi på prosjekta dei er hyra inn til å løyse. I staden for å velge det mest praktiske, og det som gir best resultat for brukaren, så vel dei det som dei trur vil gjere dei sjølve meir kompetente og attraktive på arbeidsmarknaden seinare.

    Dømet på det sistnevnte der eg jobbar, er at ein har valgt å implementere alle nye interne applikasjonar som web-løysingar. Altså, at alle brukargrensesnitt-flatene blir bygde opp via HTML + JavaScript (via JavaServer Faces).

    Dette er eit val som gjer konsulentane meir attraktive på arbeidsmarknaden, pga. at det er stort behov for utviklarar som beherskar utvikling av web-løysingar. Samstundes gjer det at brukaropplevelsen til løysingane synk med 40-50%, pga. at det er til dels sterkt begrensa kva ein kan gjere i HTML + JavaScript kontra i meir modne GUI-rammeverk.

    Altså, at ein nyttar feil teknologi, fordi ein har lyst til å lære seg det. Og ein slepp unna med det, fordi leiarane i det offentlege ikkje er kompetente nok til å kunne overprøve dei råda som konsulentane kjem med.

    I tillegg er det masse eksempel på at ledelsen lar seg dupere av selgarar frå multinasjonale selskap (*host* Oracle *host*), og at dei dermed ikkje vel det dei burde ha valgt (som f.eks. Java + Tomcat/JBoss, som du nemner frå Sverige).