Assignment 2 Trade off between Model size, Prompt type, Time Taken and Quality ❖ It is mandatory to maintain a github repository for assignments since subsequent assignments will require the same files and functions for update. ❖ You need to submit a zip with name ROLL_NUMBER.zip (eg:PhDXXXXX.zip) which should have: ● A pdf which should have all your results and conclusions mentioned and link to the github repository. ● Code files in .py/.ipynb format only, colab links will not be accepted. (Download your collab file and put in zip) ● Task: ○ Three publicly available LLMs of different sizes: ■ https://huggingface.co/google/gemma-2b-it ■ https://huggingface.co/microsoft/Phi-3.5-mini-instruct ■ https://huggingface.co/meta-llama/Meta-Llama-3.1-8B-Instruct (NOTE: In case you observe that inference of 8B model is taking time you can use free api credits from https://together.ai/ , you can use these and many more similar api key exists) ○ Use this https://huggingface.co/datasets/cais/mmlu/viewer/college_mathematics dataset of Mathematics question answers to do this task. ○ Implement functions to perform inference using below type of prompts for all three LLMs. ■ ReAct Prompting For more clarity you can refer : https://www.promptingguide.ai/
