While artificial intelligence might feel like an overused buzz word that’s been latched onto by a first-year marketing major, unbelievably, there are companies out in the world such as AT&T who are applying the technology practically to produce real-world results.
Microsoft and Google may be amongst the loudest companies pitching AI like it’s the next NFT bubble, other, enterprises are more quietly applying the concepts of large language models to generate code, summarize meetings, assist in customer service, review contracts, and host competitive generative games within their company.
I had an opportunity to sit with Andy Markus, Chief Data Officer at AT&T, to discuss AT&T’s practical application of AI with its own internal pre-generative Ask AT&T platform.
ASK AT&T’s brief history
Last year AT&T went ahead and built its “own generative AI tool” for its employees to interact conversationally with and using natural language. Ask AT&T was originally built on OpenAI’s ChatGPT technology but has expanded to support other LLMs that include Meta’s LLaMA2 and Falcon Transformers.
AT&T also partnered with Microsoft to secure the data that flows through Ask AT&T.
Back in September of 2023, AT&T expanded use of its Ask AT&T platform to 30,000 of its employees and today, I’m trying to figure out how well that went as well as what it means for the company’s future.
More on ASK AT&T
Kareem Anderson: How is Ask AT&T engaged? Is it web based, is it an OS-specific app and can it be accessed across devices on internal networks?
Andy Markus: Ask AT&T is available to AT&T employees both through a web interface as well as a channel on Microsoft Teams, where employees who have access to the platform can converse with Ask AT&T just like they would with anyone else on Teams.
It’s accessible across all approved devices with appropriate security both on our internal network and externally. In addition, the bulk of Ask AT&T use occurs via API calls. The Ask AT&T API calls from other production systems exceed 20M.
KA: It is based on OpenAI’s LLM’s, but how does it truly differ from traditional rule-based systems in its semi-limited use-cases?
AM: Ask AT&T is an interoperable platform that leverages both commercial and open-source LLMs. Many of our use cases do leverage OpenAI functionality, but we have 10 LLMs available and some use case executions involve using multiple LLMs.
Ask AT&T has safeguards to ensure that proprietary AT&T data doesn’t leak out publicly and can only be accessed internally by the right folks who need it to do their jobs. More than 68,000 employees across AT&T now have access to Ask AT&T for such capabilities as code generation, meeting summarization, customer service assistance, contract understanding and action and more.
Importantly, we’ve also created a formal process for those users to propose new use cases and work collaboratively with the AT&T Chief Data Office to turn those ideas into live applications integrated into Ask AT&T.
KA: How does Ask AT&T handle user prompts and generate responses?
AM: Natively, a user can create their own prompt when using the system. We also have an extensive prompt catalog that can be accessed by domain. All aspects of a domain use case (prompts, LLM, RAG functionality, etc.) are monitored in a leaderboard graded against 1,000 Q&As developed by the domain subject matter experts.
KA: What security measures are in place to protect corporate data when using Ask AT&T?
AM: In addition to the security, we’ve built in to prevent AT&T data from being uploaded back to the public LLMs, we’ve also instituted role-based access restrictions internally around sensitive company data.
So, for example, human resources employees will have access to the detailed HR documentation and data in Ask AT&T they need to do their jobs, but finance or marketing employees will not (unless they specifically need it for their jobs). Additionally, the Chief Data Office (CDO) and the AT&T Generative AI Governance Forum, in collaboration with AT&T’s legal experts, have created the AT&T AI Policy, which all Ask AT&T users have access to and are expected to abide by.
KA: What role does Microsoft play in ensuring the security and functionality of Ask AT&T? Are there ongoing communications with Microsoft or OpenAI about future models?
AM: We’ve worked with Microsoft to make Ask AT&T secure and safe for our employees and our corporate data. It runs in an AT&T-dedicated Azure tenant that’s been pressure tested for leakage.
We’re in conversations with almost everyone creating both commercial and open source LLMs and SLMs (small language models) and working closely with chipmakers to build and customize LLMs. Different models are suited for different applications and have different cost structures, and we’re building that flexibility and efficiency from the ground floor.
KA: How has Ask AT&T tangibly contributed to improving employee productivity and efficiency?
AM: Coding was one of the first use cases we tested Ask AT&T on, and it’s been a great aid on that front, with developers coding anywhere from 30% to 50% faster. We’ve also seen great results with a GenAI-powered vulnerability remediation tool that we created.
This tool can detect issues in our software code and, crucially, write and automate patches and fixes as they occur. While people are always in the loop to monitor and refine, we’ll be able to reduce our response times from days or weeks to literal seconds. Rather than chasing bugs and issues, our developers will be able to focus on creating new apps and services.
KA: Are there plans to expand Ask AT&T to other departments or functionalities beyond coding and software development?
AM: Absolutely. Another exciting capability we’re developing is to use Ask AT&T to talk to and analyze data – we call this Ask Data. This functionality will query data – automatically detecting fields, joining tables, and creating the code to help us gain insights from the vast data flows we manage on our network. As we say internally, human language is the new SQL or new Python.
This is all opening new opportunities to reimagine how we execute the business, in everything from analyzing customer needs to optimizing where we place radios and other cellular equipment. More broadly, we’re training it on our contracts and financial materials, training it to assist with our human resources questions, to help make our customer service agents even more efficient so that they address customer questions better and faster and we’re training it to help identify and fight off fraud attempts.
KA: What sets Ask AT&T apart from other AI tools used within AT&T?
AM: AT&T is a longtime user and even pioneer in AI. In fact, in 1955, our researchers helped organize and were key contributors at the conference where the term “artificial intelligence” was coined. More recently, we’ve been applying AI across AT&T to provide improved service and value to our customers, increase operational efficiencies, and drive new revenue opportunities for the company.
AI is imbedded across AT&T. Customer service is one major area that’s benefited. Behind the scenes, AI optimizes the daily routes our field technicians take in their trucks to serve more customers and handle more repairs with less fuel consumption. And AI helps us recognize and block fraud in the network in near real time to reduce the number of spam calls our customers receive.
What’s new with GenAI tools like Ask AT&T is that they make AI accessible to everyone. Everyone can integrate AI into their daily workflow with zero coding or software development background. This is a huge shift, similar to how desktop PCs made computing available to everyone.
KA: How does Ask AT&T to integrate with existing workflows and processes?
AM: Ask AT&T can be an assistant to almost every employee in a variety of ways. For example, employees can feed transcripts of calls and meetings into Ask AT&T, and it will almost instantly summarize the discussion and include key action items, cutting down on the number of people who need to join a meeting in person. Employees can also upload their own documents into Ask AT&T and query those documents, get summaries, and more.
Many production uses of Ask AT&T involve API access. And as Ask AT&T and GenAI take off, we’re always mindful that humans remain in the loop and remain responsible for using this technology appropriately and ethically while safeguarding our customers and our data.
Using ASK AT&T Competitively
In addition to leveraging AI for commercial purposes, AT&T is also using LLMs to host its TDP’s AI Learning & Problem Solving Challenge.
AT&T recently challenged around 700 corporate systems employees to pair off into 17 teams each with 70 people full to use its Ask AT&T AI platform to surface innovation for the company’s future through a Technology Development Program competition.
The goal of TDP’s AI Learning & Problem Solving Challenge is to find the most creative ways to “solve real-world work challenges.”
Many teams were located in the same place, and it was not uncommon to see conference rooms buzzing with activity.
Mart Hoover, Associate Director of Technology II Corporate Systems, AT&T
Team members would gather, brainstorming ideas, refining their pitches, and conducting practice presentations. It was a testament to the collaborative spirit of the competition.
In the end, AT&T’s PLEdge of Progress team presented a conceptual solution using Ask AT&T to address issues of guidance, time, and motivation. Ultimately, PLEdge of Progress came up with microlearning tools the leverage engaging videos, gamification and customized learning paths produced through analyzed data.
Ask AT&T aided in the overall presentation of the ideas and concepts PLEdege offered by researching industry trends, drafting business plans, conducting SWOT analysis and the production of PowerPoint templates for pitches.
It was an awesome experience to be on such an innovative team. Innovation challenges like this one are a great way to step out of the regular workday and truly push yourself to learn and innovate quickly.
Brian Bell, corporate systems associate director of technology II
For those interested, AT&T also issued the information of the second and third place that included a four-tiered strategy to improve PLE and filters to match specific learning styles, respectively.
The question of a potential AI bubble remains unanswered as more companies rush to market themselves alongside the pre-generative craze but companies like AT&T are at least securing their futures with less grandiose claims and more practical applications of the technology.
