5 tips for building foundation models for AI

Many businesses are just beginning to grapple with the impact of artificial intelligence, but some have been using machine learning (ML) and other emerging technologies for over a decade.

Also: Most AI projects are abandoned – 5 ways to ensure your data efforts succeed

For Manish Jethwa, CTO at Ordnance Survey (OS), the UK’s national mapping service, the priority is to combine his organization’s AI and ML experiences with recent advances in generative AI to refine, distribute, and apply its treasure troves of data.

Jethwa explained to ZDNET how language models (LLMs) are helping OS users find and query geospatial data. One of the key elements here is the organization’s foundation models for AI, which serve as a base for building more specialized applications.

Also: 4 questions to ask yourself before betting on AI in your business – and why 

While tech analysts like Gartner suggest there is much conjecture about whether business leaders should buy or build AI models, Jethwa and his team at OS combine foundation models with commercially available tools to exploit and distribute geospatial data.

Here are five key lessons that business leaders can learn from Jethwa’s deployment of foundation models for AI.

1. Develop a strong use case

Jethwa said OS is developing foundation models to extract environmental features for analysis in a copyright-sensitive manner.

“Many of the existing models trained by the large technology organizations will be based on commercially available data,” he said.

OS benefits from a long history of high-precision data collection that feeds the organization’s AI developments.

“Where we’re trying to extract features, we build foundation models from the ground up,” he said. “That will be a model where we’re defining the full training set with the labelled data that we’ve got internally.”

Also: 4 ways to turn AI into your business advantage

The foundation models are also used as the basis for data analysis in other areas. Jethwa said the message here is simple: you can use what you’re already built again and again.

“The foundation models are there to help us build subsequent output. So, if we then wanted to learn about roof materials or green spaces or biodiversity, we could do all of that from the same foundation model,” he said. “Rather than having to train multiple foundation models, you just do the fine-tuning at the end. This process allows us to connect to the problem we’re trying to solve with source data.”

2. Establish purposeful methods

Jethwa said focused training helps constrain costs when building foundation models.

“We have to be mindful that, when it comes to training these models, we’re doing it purposefully, because you can waste a lot of cycles on the exercise of learning,” he said. “The execution of these models takes far less energy and resources than the actual training.”

OS usually feeds training data to its models in chunks.

“Building up the label data takes quite a lot of time,” he said. “You have to curate data across the country with a wide variety of classes that you’re trying to learn from, so a different mix between urban and rural, and more.”

Also: 5 ways to be a great AI agent manager, according to business leaders 

The organisation first builds a small model that uses several hundred examples. This approach helps to constrain costs and ensures OS is headed in the right direction.

“Then we slowly build up that labelled set,” Jethwa said. “I think we’re now into the hundreds of thousands of labelled examples. Typically, these models are trained with millions of labelled datasets.”

While the organization’s models are smaller, the results are impressive.

“We’re already outperforming the existing models that are out there from the large providers because those models are trained on a wider variety of images,” he said. “The models might solve a wider variety of problems, but, for our specific domain, we outperform those models, even at a smaller scale.”

3. Use other LLMs for fine-tuning

Just because OS uses its own foundation models doesn’t mean the organisation ignores well-known large language models, said Jethwa: “We’re building off the existing models and doing the fine-tuning based on our documentation.”

OS uses the full breadth of commercially available LLMs. As a Microsoft shop, the organization uses Azure machine learning models, Python-based tools, and other specialist capabilities.

Also: ChatGPT isn’t just for chatting anymore – now it will do your work for you

Jethwa said OS also explores partnerships with external organizations, such as IBM and other technology suppliers, to generate collaborative solutions to data-led challenges.

Once again, just like with foundation models, the aim is to keep costs constrained.

“It’s an effort to rationalize,” said Jethwa. “Internally, the main way of taking that approach is by building up slowly and ensuring the destination you’re trying to head towards is achievable, and you’re not wasting resources with fruitless activity.”

4. Think about commercialization

Now that OS has started to build and refine its foundation models, could these technologies be used by or sold to other organizations? The answer, said Jethwa, is possibly.

One of the key issues is Crown copyright, a form of copyright that applies to assets created by UK public sector employees.

“I think there will be opportunities for us to share those foundation models at some stage, but the fact that they’re built on Crown copyright means we’re still trying to understand the potential impact of doing that work externally,” he said. “There are challenges there around giving away the crown Jewels — these assets are, quite literally, Crown copyright jewels, so we’ve got to be careful.”

Also: Microsoft is saving millions with AI and laying off thousands – where do we go from here?

When OS does provide open access, Jethwa said the organization’s assets mustn’t be collected and monetized without producing benefits for UK taxpayers.

“We’re trying to protect our data as much as possible, but at the same time, deliver as much value for the UK. So, it’s trying to get that balance right, which is a challenge.”

5. Keep one eye on the future

Jethwa said his organization’s work on foundation models has proven the benefits of generative AI for opening access to in-depth insight.

“It’s provided that key unlock, whereas previously, you always felt as though that access was slightly out of reach in terms of how you might perform the interaction, get to the data, and refine the request.”

He painted a picture of how the OS approach to AI might develop over the next decade.

“I can imagine an interface where there’s a map and you can say, ‘I’m interested in this area,’ and you can zoom in and the AI will ask, ‘What specific things are you looking for?’ When you say ‘schools,’ the AI will ask what types of schools, and you’ll have that dialog back and forth via the interface.”

Also: 5 entry-level tech jobs AI is already augmenting, according to Amazon

Jethwa said the key to long-term success is using APIs and data to create definitive answers to prompts using trusted sources, including OS information combined with external sources.

“AI models are great in terms of aggregation and a probabilistic view, but, in our example, you don’t want to know probabilistically where the schools are,” he said. “You want to know where the actual schools are. AI has to translate a true request, going back to an authoritative source, which OS is, and we can pull the data and deliver the output.”

Get the morning’s top stories in your inbox each day with our Tech Today newsletter.

Keep your entertainment at your fingertips with the Amazon Fire TV Stick 4K! Enjoy streaming in 4K Ultra HD with access to top services like Netflix, Prime Video, Disney+, and more. With an easy-to-use interface and voice remote, it’s the ultimate streaming device, now at only $21.99 — that’s 56% off!

With a 4.7/5-star rating from 43,582 reviews and 10K+ bought in the past month, it’s a top choice for home entertainment! Buy Now for $21.99 on Amazon!

Help Power Techcratic’s Future – Scan To Support

If Techcratic’s content and insights have helped you, consider giving back by supporting the platform with crypto. Every contribution makes a difference, whether it’s for high-quality content, server maintenance, or future updates. Techcratic is constantly evolving, and your support helps drive that progress.

As a solo operator who wears all the hats, creating content, managing the tech, and running the site, your support allows me to stay focused on delivering valuable resources. Your support keeps everything running smoothly and enables me to continue creating the content you love. I’m deeply grateful for your support, it truly means the world to me! Thank you!