Considerations for Data Workforce 2028
This article details the author’s experience in a first-of-its-kind attempt for an Army Reservist in an Operations Research/Data Science role.
Recommendations
My time supporting the Land Warfare Laboratory’s (LWL’s) COVID-19 decision support efforts has reinforced many lessons about ‘frugal innovation’ and transient workforces. This includes the potential to rapidly scale up Defence capability at low cost and options for Data Workforce 2028. My recommendations upfront:
- Fully embrace the mindset and methods of ‘The Gig Economy’. Implement the Defence equivalent of an online job marketplace specifically for short-term and freelance analytical and forecasting tasks, not roles. Getting ad-hoc jobs done quickly and cheaply using a virtual and remote workforce, with ‘gig’ candidates stemming from anywhere in Defence.
- Using ‘no-code’ visual programming environments for general analytics and Machine Learning/Artificial Intelligence powered forecasting tasks. These developments in Data Science dramatically lower the barriers to widespread adoption within Defence.
- Adoption of Data Science within Defence happens with the mindset of a company start-up. Firstly introduce no-code forecasting technologies, then let a nascent groundswell of understanding empower the first generation of solutions, keep iterating, and only then identify roadblocks or further avenues of exploitation with structured planning. Data Science technology moves too quickly for any other viable development paradigm at reasonable costs.
Background
Integration into the LWL COVID-19 taskforce
My business experience is doing innovative Data Science very cheaply and in a highly competitive analytical environment. This includes co-founding a proprietary trading company that is at the forward edge of forecasting technologies and practices to develop and consolidate trading margins. Transient workforces and ‘frugal innovation’ have been central to this work.
Like many companies, there was a massive disruption in our chosen markets as COVID-19 was taking hold in early March. Many of our niche markets suspended trading due to thin trading volumes, which in turn caused decisions regarding trading operations. The decision was made to suspend trading, and for the first time since I had co-founded the company over seven years ago, I suddenly found myself with a lot of spare capacity.
I reached out to a contact I had made at the LWL some years ago, and half-jokingly expressed the situation of my new found spare time, and that it could be put to some use. I didn’t expect a positive response. To my shock and delight, I was offered an opportunity to apply my knowledge and experience in Machine Learning / Artificial Intelligence (ML/AI) and contribute to the COVID-19 forecasting efforts of the LWL.
I had made a heart wrenching decision to cease Active Reserve service as an Infantry Section Commander after 15 years to give the time and effort to our business. The LWL role was an opportunity to return to the Army Reserve after having been inactive for nearly two years.
Conduct of tasks within the LWL’s COVID-19 taskforce
As soon as I was integrated into the team, I hit the ground running with my first task to create detailed country growth projections of COVID-19 infection rates and death rates. Apart from being given the high level intent of the work, I was left to my own devices and worked remotely.
I developed my own dataset that housed over 900 variables that could potentially relate to predicting COVID-19 and began to generate my first analytical product four days after. After another week of development, the structure of an analytical report housing detailed COVID-19 projections of Australia and South-East Asia was finalised. These reports are still maintained for contribution to bi-weekly publishing windows by the LWL.
After this first task was finalised, a new task was given which greatly ramped up the level of difficulty. The high level intent was to develop forecasting insight regarding potential COVID-19 impacts on Australia’s Near North via data science driven techniques. I assessed the problem and concluded that an all-encompassing series of geopolitical forecasting solutions should be developed. COVID-19 would act as one factor that could be experimentally perturbed and resultant outcomes analysed.
The third and final task was a decision support guide relating to defining optimal paths for an escalating series of measures to combat COVID-19. Optimal paths were determined by examining intervention decisions and their efficacy respectively. It also included examining the entirety of decision chains to define an optimal chain for escalation.
As sudden as the work ramped up for these three tasks, the work ramped down. At the time I had committed just over 30 working days. This suited me just fine, as our business at the same time was ramping up again with our chosen traded markets spooling up in the right way to recommence trading.
Immediate lessons learnt
LWL feedback suggested my contribution made a difference. I was rapidly taken out of SERCAT 3, and handed a short term DA-26 contract to work on Reserve days. Very quickly, I was delivering a significant new capability. I was glad to be ‘back in the system’ and contributing to Defence in a meaningful way. From a commercial standpoint, Defence was getting fantastic value for its expenditure, and this leads to a key observation regarding transient workforces for Defence in the data science context:
If someone has the right skills and wants to contribute to data science in Defence, the chances are that their market rate in private industry is multiples of what Defence can afford.
Contract daily rates for highly proficient data scientists in private industry range from anywhere between $2K and $4K per day. Defence was onto a bargain when it is paying an Infantry Corporal’s daily rate of $217 for my skillset.
This leads to another crucial point I have learnt in business regarding hiring in the data science context:
Talented people will work for a fraction of their market rate if it suits them, and if they find the work interesting.
Transient labour and The Gig Economy
The whole premise of ‘The Gig Economy’ is to harvest time and commercialize unconventional levels of availability. Whether it be intermittent, or surge-like and spasmodic, the potential for unlocking freelance labour in today’s labour market squarely involves the ability to harvest these oddly shaped labour opportunities. The gig worker generally understands that they will operate at a discount to conventional rates given they have much more control on how they deliver on a product or service. This factor combined with some sort of interest in the industry or subject matter by the freelancer, and I have found tasks can be readily achieved at 10% to 20% of normal costs.
Harvesting transient labour and the frugal innovation that transient workforces entail are two sides of the same value-add coin.
Our company has a small permanent labour footprint, and ever since its inception, our business has been heavily supported by virtual freelance labour. I have personally instigated and managed over 100 ‘gigs’, 80% of which were under $1,000 in cost. At the beginning it was an almost revolving door, but we have now settled on a constellation of virtual freelancers that buttress our permanent labour force: we know the right person for an ad-hoc task, and even have redundancy options if our first choice cannot be available.
Virtual short-term project teams
My freelancing and virtual workforce management experience also led to another level of scale: an engagement model which Defence would be familiar with, but perhaps not in the sense that our business approached it: a virtual short-term project team, as opposed to physical co-location of team members working together on short-term contract.
We have recently completed a two year development project to build our own bespoke proprietary trading platform from scratch. Team members operated in a mix of home locations and office spaces in their home countries. This involved wrapping a core of advanced ML/AI capabilities with low latency trading execution technology. All in all, it was a complex fusion of ML/AI and conventional IT.
We simply couldn’t afford to pull off this development objective in any conventional sense, and so I was left with no choice but to manage our stakeholders and project team members across six different countries and five different time zones spanning the globe. The only member to meet all other project personnel throughout this two-year project lifecycle was myself.
The end result of trying to frugally innovate out of necessity are multiple estimates of a $3M to $4M asset valuation on the finalized capability; our total project spend had been $500K. Admittedly, we had to wade through a web of additional risks to achieve our end goal, this also notwithstanding the project timeline; a physically co-located team would have achieved the goal likely six months faster. Yet in terms of business impact, a virtual workforce working in disparate locations and time zones achieved in delivering a significant capability at a fraction of the price.
There could be potential in Defence for development of virtual workforces in a similar context: organizational capability is developed where time is not a critical factor, and where the vast majority of personnel working in physically disparate locations across the country never co-locate. More specifically, smaller tasks can be posted as one-off jobs for internal application over the Defence equivalent of a virtual freelancing hub environment which connects anyone in Defence with the right clearance to the job opening.
‘No-code’ visual programming environments for analytics and ML/AI
No-code visual programming environments dramatically lowers the barriers to developing data analysts with ML/AI enhanced forecasting abilities. The competitive labour landscape for highly skilled data scientists will have Defence finding it difficult to compete for top tier talent. But there may be a lack of understanding of just exactly what is required in terms of forecasting capability, both in personnel and technology.
Summing up my view using an analogy:
Defence needs pilots. Not physicists who intimately understand the physics of flight.
After seven years of using ML/AI personally, I’m very much an experienced pilot of this technology, but I have little ability to construct my own algorithms from fundamental first principles.
The vast majority of ML/AI exponents do not write the ML/AI algorithms that they use. Only a select few people have the ability to develop algorithms from the fundamental principles of mathematics and computer science that they are inspired from. ML/AI users initially interacted with these forecasting technologies through object orientated programming, which unlocked the first tranche of analysts outside of the original core of select individuals. Programming, whilst opening up ML/AI for many IT professionals, still leaves a lot of potential users without an ability to interact with this burgeoning new technology.
The seed of a ‘no-code revolution’ was planted over a decade ago with the first ever attempts to allow non-programmers to manipulate algorithmic code as building blocks in a visual environment. I was a prime candidate for this ‘no-code revolution’: I had a Master’s in Applied Statistics but no formal IT background, and hated any kind of programming I had to do in my Master’s course!
I found I could readily understand the concepts behind the algorithms I was trying to use practically given my statistics background. Arguably I was even over-qualified. After using my spare personal development time whilst on deployment in Timor Leste in 2012 to learn about no-code ML/AI environments, in early 2013 I took a short two week ML/AI Analyst course in Germany to consolidate my self-learning in using a particular free, open-source ML/AI visual coding environment. In a matter of days, my cohort and I were developing our own powerful forecasting solutions. I was the exception in my cohort, other personnel were business analysts with no quantitative background. They too walked out of this short course armed with a practical understanding of how to bring ML/AI enhanced analytics back to their organizations.
For my own personal use case, I can emphatically say that our business as it stands today originated on the back of free, open source, and visual ML/AI technology. When the going got tough and our start-up capital had burnt out to no avail, that’s when we really started to learn. It was mastering free technology that steered the course and allowed us to survive and prosper. The user experience has been so positive since the initial learning, that our business is still enabling this technology through visual ML/AI environments today.
The image below is a typical example of an analyst created visual ML/AI workflow. In this example, I can see the user has created a sentiment analysis tool of cinema movie reviews. They initially clean and pre-process the data before apply a Decision Tree ML algorithm prior to finally ‘scoring’ (assessing) forecasting efficacy of this forecasting solution.
There is no technical skills based rationale that prevents a multitude of analysts with no quantitative backgrounds in Defence to master this free, open source ML/AI technology upon completion of a short analyst level course.
A few dozen analysts qualified via a short course, and potentially co-ordinating task execution virtually, and I sincerely believe there would be an analytical revolution within Defence that would generate a massive organizational impact. The applications for this technology are as far reaching as our creativity. ML/AI analyst level tasks could include:
- Optimizing recruitment: predicting the best candidates for admission into Defence
- A ‘churn’ prevention model for personnel retention: predicting personnel looking to leave Defence
- Intelligence operations using ML/AI. Automated data mining of internet data sources for predictive signals of enemy intent
- Predictive maintenance of even the lowest levels of equipment
- Inventory prediction. Leaner just-in-time inventory management using ML/AI for prediction.
Anything that has a predictive element can be enhanced by ML/AI forecasting technology, the majority of which can be developed and managed by analyst level operators with no more than timely assistance from SME’s, which would in this case be Defence’s Operations Research qualified personnel.
There also does not even need to be any sort of data basing technology for implementing analyst level solutions. A selected visual ML/AI environment in conjunction with the already existent and ubiquitous Microsoft Excel can serve very well as a substitute database. Microsoft Excel is also a relatively easy environment to learn. From my personal experience, Microsoft Excel paired with a visual analytics environment is enough to achieve desired outcomes in lower level predictive analytics tasks.
It would be the feedback loop generated from this first generation of grass-roots analyst level effort that would illuminate the development path for more structured investment in Data Workforce 2028. As analysts gain experience and expand the scope of attempting predictive solutions, they would start to appreciate any roadblocks given their bottom-up approach. From my own experience, the first roadblocks would form when trying to industrialize an existing solution. Perhaps an analyst develops an inventory prediction solution for a specific unit that warrants scaling to other parts of Defence; just how to industrialize it would be a natural first port of call in top down planning thereafter.
Conclusion
Data Workforce 2028 and the ‘Big Data’ revolution presents a remarkable opportunity to change the analytical paradigm within Defence. This includes using a variety of different engagement models, empowering established analysts within Defence and using a single new additional visual ML/AI toolset. Taking these iterative steps at low cost in the short term regarding the capture of transient labour would shed light on this first generation of analyst level predictive analytics solutions, well before 2028 even arrives.
About the author
Bret Dvoracek enlisted as a Rifleman in the Army Reserve in 2002, and was posted to The 8th/7th Battalion, The Royal Victoria Regiment for the duration of his tenure in the Active Reserve. He left the Active Reserve in 2018 to further focus on building the company he co-founded in proprietary trading in 2013. He was an Infantry Section Commander for the last 10 years of his service, and deployed on operations to Timor Leste in 2012. Outside of his military experience, he has a double degree in Business & Arts, and also holds a Master’s Degree in Applied Statistics. He has been an avid disciple of ML/AI driven Data Science ever since his personal development time in Timor Leste in 2012 led him to learning these technologies.