From the editor's desk: Intelligence is needed to train AI

22 November 2023 News

Peter Howells, Editor

As I sit in front of my computer and ponder the year that is swiftly coming to a close, I again marvel at everything that has happened. It has definitely been a busy year. There have been plenty of announcements in the tech space, flooding my inbox, especially when it comes to artificial intelligence and machine learning.

No matter who you speak to, they will have heard something about AI and the effects it will have on our everyday life. And they all have their own opinions too. Many of them, however, are not based on any sort of scientific fact.

One opinion piece that did cross my desk this month was an article on training of AI models. Many people do not realise that AI is trained on human-generated information. This mostly comes from information that already exists on the internet in some form: articles that have been written, responses to questions, shopping habits, browsing history; pretty much anything is fair game for use in the training of AI models.

But there is a finite amount of non-repetitive information that exists. Let me clarify what I mean by this. Much of the enormous amount of information created each day is a copy of data/information that already exists. People reposting text, images and other multimedia, copying of other data from one platform to another; all this counts towards the total amount of information created.

Yes, there is still a massive amount of information available for training, but AI models are becoming more and more powerful, and are able to be trained on larger volumes of data. This year, even though the total amount of data generated globally was in the order of 120 zettabytes, much of this cannot be used for training models. ChatGPT was trained on 570 gigabytes of data, which amounted to around 300 million words. The more data used to train these AI models, the more accurate the models’ responses will become.

And this is where the concern starts to kick in for AI researchers; the volume of datasets needed to train AI models is growing much more rapidly than the growth of online data stocks. If the current training trend continues, in a paper published in 2022 it was predicted that we will run out of high-quality data before 2026. If models then turn to the remaining low-quality data, this will also be exhausted, sometime between 2030 and 2050.

But do we really want to train our models on low-quality data? We all know the bad decisions that can be made when only poor-quality data is available. After all, the internet is full of examples of ‘average’ people doing stupid things based on lack of insight or forethought. Do we really want our artificial intelligences to be only as smart as the average person?

One hope is that newer AI models will have a lower data overhead, that is, to be able to be trained suitably well using less data than their predecessors. I believe this would be similar to how many people get to conclusions nowadays – they are able to make quite reasonable decisions even when they do not know everything about a subject.

The one overriding thing I have taken away from all this talk about AI during this year is that we are certainly all living in an interesting and exciting era, even if it can be quite concerning at times.

To all our readers I would like to take this opportunity to wish you all a joyous and restful season. May your new year be filled with new goals, new achievements and above all, happiness.


Share this article:
Share via emailShare via LinkedInPrint this page

Further reading:

E-mobility: Navigate safety, interoperability and conformance
Concilium Technologies News
In this whitepaper, these challenges are discussed in more detail and the question is asked: How can EV and EVSE manufacturers navigate a complex regulatory landscape and deliver a quality product, without compromising time-to-market projections?

Navigating South Africa’s transition from the NQF to OQSF framework
South Africa is at a crossroads of a significant educational and vocational transition, as it shifts from the National Qualifications Framework (NQF) to the Occupational Qualifications Sub Framework (OQSF) managed by the Quality Council for Trades and Occupations.

Practical insights: Draft regulations on proposed sectoral targets
On 1 February 2024, the Minister of Employment and Labour issued new draft regulations concerning sectoral targets (draft regulations), providing a 90-day window for interested parties to comment.

Top skills needed in microelectronics industry
The EU funded project MicroElectronics Training, Industry and Skills (METIS), which began in 2019, has officially concluded, and with this, SEMI Europe has published the final Yearly Monitoring Report.

RS expands its professional lineup
RS South Africa News
RS PRO offers a wide range of high-quality electronics components, to meet a range of needs, from industrial connectors and passives to the latest test and measurement equipment.

Alternative energy resource hub
Mouser Electronics is providing engineers with key information to solve today’s design challenges through its dedicated alternative energy resource centre.

NECTO Studio 6.0 brings circuits to life
MIKROE has added increased graphical functionality and Clang support for ARM and RISC-V to its cross-platform IDE for embedded applications.

Microchip earns certification in vehicle cybersecurity
Altron Arrow News
Microchip Technology’s corporate processes associated with specific automotive work products have recently been audited by a third party, UL Solutions, and certified as compliant to ISO/SAE 21434.

From the editor's desk: A scourge on education
Technews Publishing News
he current government has done very little to curb the rampant affect that loadshedding has had on education in the last 10 years.

SiLabs Q4 2023 results
Silicon Labs has released its fourth quarter results for 2023, indicating results above expectations, and an anticipated growth in the first quarter of 2024.