Slack, now part of Salesforce, has joined forces with Amazon SageMaker JumpStart to introduce AI services that will improve data searching, summarization, and security for users. The collaboration involves leveraging SageMaker JumpStart’s large language models (LLMs) in a way that data stays within Slack’s infrastructure and does not get shared with external model providers. The collaboration maintains Slack’s stringent security practices and compliance standards, while also using Amazon SageMaker’s inference capabilities to enhance system scalability, performance, latency, and throughput.
In simple terms, SageMaker JumpStart is a machine learning (ML) platform that enables users to quickly choose foundational models (FMs) based on predefined quality and responsibility metrics to carry out tasks like text summarization and image creation. Crucially, user data remains encrypted, private, and confidential, and is never used to train underlying models or shared with third-party vendors.
Slack has rolled out its own generative AI tool, Slack AI, to augment users’ ability to search and access large volumes of data more efficiently. Users can now ask questions in natural language and receive concise answers, consolidate channel conversations, and receive personalized daily summaries from select channels.
In the implementation, Slack selects the right FMs from SageMaker JumpStart and hosts them on its own Amazon Web Services (AWS) infrastructure, ensuring that the data sent to these models stays within its own infrastructure. Data sent for invoking SageMaker models is also encrypted for secure transactions. SageMaker JumpStart permits Slack to adhere to high standards of security and privacy while employing powerful models that optimize the performance of Slack AI.
The performance of Slack’s model deployment is further supported by utilizing SageMaker’s range of instance types tailored for different latency and scalability requirements. Slack AI can use multi-GPU-based instances to host multiple copies of a model, which helps conserve resources and minimize model deployment costs. Furthermore, Slack uses the Least Out-Standing Requests (LOR) routing strategy to evenly distribute requests to instances with high capacity, leading to a 39% reduction in late responses. Overall, this collaboration allows Slack AI to offer rich AI features swiftly and robustly while upholding the company’s trust and security standards.
From an author’s perspective, Jackie Rocca, VP of Product at Slack, has spearheaded the implementation of Slack AI. She previously worked as Product Manager at Google for over six years where she assisted in the launch and growth of Youtube TV. The team also includes members from AWS, each of whom bring their own unique expertise and experience to the project.
