The emerging importance of continuous intelligence

Enterprise digitalisation has given a boost to Business Intelligence (BI) software, as companies strive to extract the maximum commercial value from their burgeoning data assets.

The trend is compounding demand for coders who can tune BI tools for specific lines of business or data streams: 71% of respondents to a survey of software developers by Reveal expect their focus on BI to increase markedly in 2022. However, 53% of those surveyed expect that their biggest business challenge will be recruiting developers with the right skills.

In the US, figures from the Bureau of Labor Statistics report that jobs in the software test and development sector are projected to grow 22% from 2020 to 2030 – a rate that’s much faster than the average for all occupations. Finding suitably qualified people to fill those places is, of course, another matter.

Despite this, future demand for BI solutions seems assured. Verified Market Research valued the global BI software market at $18.33 billion in 2020, and forecasts it will hit $35.18 billion by 2028; Fortune Business Insights has higher expectations, and reckons the same market will be worth $43.03 billion by 2028.

Growing global market

Whether by then BI will still be known by that name is open to speculation. The nature of BI has often evolved in line with changing intelligence needs and enabling technologies, but there are indications that its evolutionary span might be winding down.

As an IT discipline, BI could be overtaken by the speed of market forces

As now applied, BI is the science of analysing data assets to derive intelligence about market and operational conditions to inform data-driven decision-making. It’s mostly applied in areas like sales, productivity, CRM and for cost reduction, as well as to identify broad trends and emergent market opportunities.

But as an IT discipline, BI could be overtaken by the speed of market forces – literally. That’s because conventional BI is reactive: it works by analysing static data sets ‘at rest’ in databases or data warehouses. This means that there’s a time lapse between when the data is collected and queried, and then outputted as actionable intelligence.

Although advanced Big Data analytics and cloud-based high-performance computer power have accelerated BI time-to-insight, user-organisations are increasingly interested in in-memory analytics techniques that scrutinise data in real-time (or near real-time) as it is streamed in.

Of the software developers surveyed by Reveal, 77% now incorporate BI apps directly into products developed for end-users. The aim is to close gaps between the point at which data is collected and at which it is processed to reveal actionable insights.

Fast-moving environment

This trend has intensified expectations of what BI can – and must – deliver to enterprise decision-makers and their line-of-business units. So, can conventional BI keep up with those expectations?

Jeffrey Shoreman, CEO at Magnitude Software, thinks not. He states

"Companies have become adept at leveraging BI tools to answer predictable questions, but in today’s unpredictable and fast-moving business environment, BI reporting is no longer fast or agile enough,”

“Achieving sustainable market advantage now goes far beyond standard metrics.”

Decision-makers need to “reach beyond the conventional deep post-hoc analysis to become data-driven throughout every aspect of the business”, Shoreman says. He is an advocate for an emergent technology called Continuous Intelligence (CI).

Continuous intelligence

Gartner defines CI as a process in which real-time analytics are integrated into business operations, processing current (and historical) data to prescribe actions in response to business moments and other events.

As such, it is programmed to make connections that alert businesses to patterns that could denote potential business opportunities. Some types of CI incorporate AI and Machine Learning to inform predictive analysis and automate decision-making support.

CI requires in-memory processing to optimise performance and enable real-time responses. More importantly, it requires human know-how to set-up and fine-tune its functionality, thus creating career opportunities for software developers across multiple disciplines.

According to Orbis Research, companies globally are increasingly turning towards CI as a replacement to BI processes.

Gartner has predicted estimable market growth for the CI sector – by the end of 2022, more than 50% of major new business systems will incorporate CI that uses real-time data to improve decisions. “CI represents a major change in the job of the data and analytics team,” according to Gartner Research VP Rita Sallam. “It could be seen as the ultimate in operational BI.”

If Sallam is correct, it’s a change that, as organisations transition their intelligence-gathering software strategies, would have a fundamental impact on the careers of BI practitioners and early-career software developers looking to specialise in BI.