Recent advances in Artificial Intelligence (AI) and Machine Learning (ML) are rocking the world— open-source Large Language Models (LLMs) and the like are democratizing models like ChatGPT. But where do we stand in terms of productionizing these models for on-device use or mobile devices? We delve into this topic based on our own insights in the industry.