Mixbook, the number one rated photo book service in the US, has harnessed the capabilities of generative artificial intelligence (AI) in Amazon Web Services (AWS) to make personalized photo book experiences. User photos are interpreted and creatively enhanced with Mixbook Smart Captions. The service does not fully automate the creative process, but guides the users’ storytelling for personalized, enriching experiences.
Mixbook has strategically transitioned operational workloads to AWS. The storage and networking capabilities of AWS coupled with an efficient, scalable, and secure storage for media file objects through Amazon Simple Storage Service (Amazon S3) have given Mixbook an operational edge. The end-product is a system characterized by reliability, superior performance, and operational efficiency.
A user uploads photos into Mixbook, which are stored in Amazon S3. The intake process involves AWS Fargate for Amazon ECS, a convenient orchestrator for containerized workloads. The system then moves into the inference stage, where it extracts essential contextual and semantic elements from the input. This includes image descriptions, temporal and spatial data, facial recognition, emotional sentiment, and labels using Amazon Rekognition. This detection is crucial for automatic photo placement and cropping, and for setting the tone of the storytelling.
The last stage is caption generation, powered by a Llama language model. The generated captions can also be edited by users, enhancing personalization. The captions are then stored in Amazon Relational Database Service (Amazon RDS).
Mixbook also ensures safety by identifying potential objectionable content with Amazon’s Rekognition. The new AWS generative AI capabilities have helped Mixbook push their creative boundaries.
The insights from a successful start with AWS generative AI solutions in 2023 have significantly improved customer experiences, given that storytelling can be a time-consuming process. Customers have found value in the unique ability of these AI-generated captions to express emotions and experiences. Mixbook hopes to continue to impress customers with continuous product development, testing, and integration.
The co-authors of this blog post highlight the effectiveness of Mixbook’s model of using generative AI to facilitate storytelling. Vlad Lebedev is a Senior Technology Leader at Mixbook; he draws on over a decade of hands-on experience in web development, system design, and data engineering. DJ Charles, the CTO at Mixbook, has a 30-year career history of architecting interactive and e-commerce designs for top brands. Malini Chatterjee is a Senior Solutions Architect at AWS, with expertise in Data Analytics and Machine Learning. Jessica Oliveira, an Account Manager at AWS, provides guidance and support to Commercial Sales in Northern California. She is particularly invested in building strategic collaborations to ensure her customers’ success.
