insight platform & tech forecasting
Determining if and when emerging technologies will become mainstream is crucial for businesses looking to invest wisely in new innovations. The "Insight Platform" is a data-centric proof-of-concept designed to forecast the future trajectories and impacts of these technologies. It aids in identifying opportunities and threats that could influence our business, empowering decision-making with greater confidence. This platform enhances strategic planning across various sectors of the business, making it more efficient, cost-effective, and informed. By leveraging vast amounts of open-source data, I led a team of data scientists, designer, front-end and back-end developers to develop an inhouse Insight Platform which is used accurately predict the likelihood of a technology going mainstream in the near future.
*note. Proposed model F1 score > 70%
Each year, companies like Gartner and Forrester continuously surveys and scouts for emerging technologies to predict the next big thing. Combining data and hunches, Gartner analysts position the range of 30-40 emerging technologies in ‘Hype Cycle’ to forecast the technology maturity.
Companies continuously improve the breadth and depth of their technology forecasting coverage, for example, in 2022, Gartner published their first Hype Cycle for emerging technologies in finance in 2022.
In the Hype Cycle report, various technology including blockchain, causal AI, Digital Twin, were reported with different levels of maturity across the Hype cycle with likelihood of whether these technologies will become mainstream in next couple of years. The insights, though valuable, is confounded with biases and hunches.
The Insight Engine was therefore developed with one mission in mind — to propose a data-driven, accurate technology forecasting model with open source data.
The project team collected a broad source of predictors, including patent and non-patent data, and experimented with various Machine Learning models to optimize the F1 score of the prediction model. We managed to arrive F1 score > 70% with our Data AugmentatiDeep Learning model using patent related data, such as applied and granted patents number counts, backward and forward citations, unique assignees etc. The results are satisfying and allowed us to make use of our model to guide our strategic innovation investment.
the born of insight engine GUI
Next, the team developed a simple and intuitive web-app for internal team to continuously consume the data and results of the tech forecasting.