Building Reflection-Gemma-2–2b | به سمت هوش مصنوعی

نویسنده(های): آرتور لاگاچری

در ابتدا منتشر شد به سمت هوش مصنوعی.

در دنیای هوش مصنوعی، توانایی مدل ها در تفکر و تصحیح اشتباهاتشان بسیار مهم است. این مقاله به بررسی این موضوع می‌پردازد که چگونه Gemma 2–2b می‌تواند مهارت‌های حل مسئله خود را بازتاب داده و بهبود بخشد.

بازتاب چگونه کار می کند؟

در طول استنباط:

• ابتدا، مدل یک استدلال در داخل ایجاد می کند برچسب ها
• سپس اگر مدل خطایی پیدا کند، اشتباه موجود در آن را تصحیح می کند هر چند بار که لازم است تگ کنید.
• برای تکمیل مدل، خروجی را در
  برچسب ها

مثال:

### Question
Jungkook is the 5th place. Find the number of people who crossed the finish line faster than Jungkook.
### Answer

The question asks how many people finished before Jungkook. If he is in 5th place, it means 4 people finished ahead of him.


It's important to understand that place numbers start from the first person to finish. So, 5th place means 4 people finished before him.

4 people crossed the finish line faster than Jungkook.

بازتاب تنظیم

ابتدا به مقداری نیاز داریم مجموعه داده ها برای آموزش جما. من چهار را انتخاب می کنم مجموعه داده ها:

• mahiatlinux/Reflection-Dataset-v2
• حرشکمر/اورکا-ریاضی-کلمه-بازتاب
• gabrielmbmb/distilabel-reflection-tuning
• Replete-AI/Replete-Reflection

اولین مورد از 9.17 هزار ردیف تشکیل شده است که در دو ستون، اعلان و پاسخ ها سازماندهی شده اند. سوال مانند “توضیح مفهوم …” است.

دومی از 2.44 هزار ردیف مسائل ریاضی تشکیل شده است.

و غیره…

ابتدا با دانلود آنها شروع کردم.

from datasets import load_dataset
dataset1 = load_dataset("mahiatlinux/Reflection-Dataset-ShareGPT-v2")["train"]
dataset2 = load_dataset("gabrielmbmb/distilabel-reflection-tuning")["train"]
dataset3 = load_dataset("Replete-AI/Replete-Reflection")["train"]
dataset4 = load_dataset("mahiatlinux/Reflection-Dataset-v2")["train"]
dataset5 = load_dataset("Harshkmr/orca-math-word-reflection")["train"]

و اکنون برای اینکه بتوانیم مدل خود را آموزش دهیم، باید دستور و پاسخ را به دنبال الگوی prompt Gemma2 در یک ردیف قرار دهیم.

user
Write a hello world program
model
```python
print("Hello World")
```

import os
import pandas as pd
data = {"text":[]}
### mahiatlinux/Reflection-Dataset-ShareGPT-v2
for i in range(len(dataset1)):
user = f"""user\n{dataset1[1]["conversations"][0]["value"]}"""
llm = f"""model\n{dataset1[1]["conversations"][1]["value"]}"""
data["text"].append(user+llm)
### gabrielmbmb/distilabel-reflection-tuning
prompts = dataset2["instruction"]
responses = dataset2["generation"]
for i in range(len(prompts)):
user = f"""user\n{prompts[i]}"""
llm = f"""model\n{responses[i]}"""
data["text"].append(user+llm)
### Replete-AI/Replete-Reflection
prompts = dataset3["input"]
responses = dataset3["output"]
for i in range(int(len(prompts)/3)):
user = f"""user\n{prompts[i]}"""
llm = f"""model\n{responses[i]}"""
data["text"].append(user+llm)
### mahiatlinux/Reflection-Dataset-v2
prompts = dataset4["prompt"]
responses = dataset4["response"]
for i in range(int(len(prompts)/3)):
user = f"""user\n{prompts[i]}"""
llm = f"""model\n{responses[i]}"""
data["text"].append(user+llm)
### Harshkmr/orca-math-word-reflection
prompts = dataset5["question"]
responses = dataset5["Reflection"]
for i in range(len(prompts)):
user = f"""user\n{prompts[i]}"""
llm = f"""model\n{responses[i]}"""
data["text"].append(user+llm)
### save as csv
data = pd.DataFrame.from_dict(data)
print(data)
try:
os.mkdir("/kaggle/working/data")
except:
pass
data.to_csv('data/dataset.csv', index=False, escapechar='\\')

اکنون مجموعه داده ای را داریم که می توانیم نصب کنیم AutoTrain که کتابخانه ای است که برای آموزش مدل استفاده کردم. آموزش هر مدل Huggingface را تنها با یک دستور مدیریت می کند.

!pip install -U autotrain-advanced
!autotrain setup

پس از نصب، می توانیم تنظیمات را انجام دهیم.

import os
project_name = 'Reflection-Gemma-2-2b' # @param {type:"string"}
model_name = 'Arthur-LAGACHERIE/Gemma-2-2b-4bit' # @param {type:"string"}
push_to_hub = True # @param ["False", "True"] {type:"raw"}
hf_token = "hf_token" #@param {type:"string"}
hf_username = "Arthur-LAGACHERIE" #@param {type:"string"}
unsloth = False # @param ["False", "True"] {type:"raw"}
learning_rate = 1e-4 # @param {type:"number"}
num_epochs = 1 #@param {type:"number"}
batch_size = 2 # @param {type:"slider", min:1, max:32, step:1}
block_size = 1124 # @param {type:"number"}
trainer = "sft" # @param ["generic", "sft"] {type:"string"}
warmup_ratio = 0.1 # @param {type:"number"}
weight_decay = 0.01 # @param {type:"number"}
gradient_accumulation = 4 # @param {type:"number"}
mixed_precision = "fp16" # @param ["fp16", "bf16", "none"] {type:"string"}
peft = True # @param ["False", "True"] {type:"raw"}
quantization = "int4" # @param ["int4", "int8", "none"] {type:"string"}
lora_r = 16 #@param {type:"number"}
lora_alpha = 32 #@param {type:"number"}
lora_dropout = 0.05 #@param {type:"number"}
merge_adapter = "true"
os.environ["HF_TOKEN"] = hf_token
os.environ["HF_USERNAME"] = hf_username
conf = f"""
task: llm-{trainer}
base_model: {model_name}
project_name: {project_name}
log: tensorboard
backend: local
data:
path: data/
train_split: train
valid_split: null
chat_template: null
column_mapping:
text_column: text
params:
block_size: {block_size}
lr: {learning_rate}
warmup_ratio: {warmup_ratio}
weight_decay: {weight_decay}
epochs: {num_epochs}
batch_size: {batch_size}
gradient_accumulation: {gradient_accumulation}
mixed_precision: {mixed_precision}
peft: {peft}
quantization: {quantization}
lora_r: {lora_r}
lora_alpha: {lora_alpha}
lora_dropout: {lora_dropout}
unsloth: {unsloth}
merge_adapter: {merge_adapter}
hub:
username: ${{HF_USERNAME}}
token: ${{HF_TOKEN}}
push_to_hub: {push_to_hub}
"""
with open("conf.yaml", "w") as f:
f.write(conf)

و اکنون فقط باید این دستور ساده را برای آموزش مدل خود اجرا کنیم.

!autotrain --config conf.yaml

و 4 ساعت بعد، آماده است که مدل ما به خوبی تنظیم شده است. 👍

استنتاج

اکنون ما مدل دقیق خود را داریم که می توانیم آن را برای حل مشکلات آزمایش کنیم.

!pip install -q bitsandbytes
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline
import torch
from transformers import TextIteratorStreamer
from threading import Thread
device = 'cuda' if torch.cuda.is_available() else 'cpu'
model_id = "Arthur-LAGACHERIE/Reflection-Gemma-2-2b"
tokenizer = AutoTokenizer.from_pretrained(model_id)
model = AutoModelForCausalLM.from_pretrained(model_id)
prompt = """
### System
You are a world-class AI system, capable of complex reasoning and reflection. 
Reason through the query inside  tags, and then provide your final response inside 
 tags. 
If you detect that you made a mistake in your reasoning at any point, correct yourself inside  tags.
Try an answer and see if it's correct before generate the ouput. 
But don't forget to think very carefully.
### Question
The question here.
"""
chat = [
{ "role": "user", "content": prompt},
]
question = tokenizer.apply_chat_template(chat, tokenize=False, add_generation_prompt=True)
question = tokenizer(question, return_tensors="pt").to(device)
streamer = TextIteratorStreamer(tokenizer, skip_prompt=True)
generation_kwargs = dict(question, streamer=streamer, max_new_tokens=4000)
thread = Thread(target=model.generate, kwargs=generation_kwargs)
# generate
thread.start()
for new_text in streamer:
print(new_text, end="")

مشکلات

بیایید ببینیم که آیا می تواند برخی از مشکلات را حل کند.

1.

The length of one span of Jinseo is about 12 centimeters (cm). 
When Jinseo measured the length of the shorter side of the bookshelf, 
it was about two spans. How many centimeters (cm) is the short side of the 
bookshelf?

 
To find the length of the shorter side of the bookshelf, we need to figure out how many spans are in the shorter side. We know that one span is 12 cm long. The shorter side is two spans, so we need to multiply the length of one span by 2. 
 
The shorter side of the bookshelf is 2 * 12 cm = 24 cm. 
 

👌 باشه اول با موفقیت گذشت. (اما خیلی ساده بود)

2.

How many diagonals can you draw in a decagon?

 
A decagon has 10 sides. A diagonal is a line segment that connects two non-adjacent vertices. 
To find the number of diagonals, we can use the formula for the number of diagonals in a polygon. 
The formula is n(n-3)/2, where n is the number of sides. 
So, for a decagon with 10 sides, the number of diagonals would be 10(10-3)/2. Let's calculate that. 
 
The number of diagonals in a decagon is 10(10-3)/2 = 10(7)/2 = 35 diagonals. 
 

✅ همچنین درست است.

3.

What is the difference between the largest number and the smallest number 
that can be made by selecting four from the number magnets 
2, 0, 3, 5, and 8 and arranging them in a row to form a four-digit number?

To find the largest and smallest numbers, I need to consider the possible arrangements of the numbers 2, 0, 3, 5, and 8. I should also consider the range of values these numbers can take. I need to figure out the largest and smallest possible four-digit numbers that can be formed. 

The largest number that can be made is 8350, and the smallest number is 0235. 

❌ دروغ است.

آخرین مورد:

4.

This year, the age difference between Minsu and his mother is 28 years old, 
and after 13 years, the age of his mother will be twice that of Minsu. 
Find the age of Minsu this year.

Let's break down the problem step-by-step:
* **Let Minsu's current age be 'm'**
* **Let his mother's current age be 'M'**
We know:
* M - m = 28 (age difference is 28 years)
* M + 13 = 2(m + 13) (mother's age in 13 years will be twice Minsu's age in 13 years)
Now we have two equations, and we can solve for our two unknowns. 

Let's solve for 'm' (Minsu's current age) and 'M' (his mother's current age).
From the first equation, we can express 'M' in terms of 'm':
M = m + 28
Substitute this value of 'M' into the second equation:
(m + 28) + 13 = 2(m + 13)
Simplify and solve for 'm':
m + 41 = 2m + 26
15 = m 
Therefore, Minsu is currently 15 years old. 

✅

پایان

امیدوارم از این مقاله لذت برده باشید و اگر اینطور است می توانید آن را کف بزنید. (شما هم می توانید من را دنبال کنید) = ).

Arthur-LAGACHERIE/Reflection-Gemma-2-2b · صورت در آغوش گرفتن

ما در سفری برای پیشرفت و دموکراتیک کردن هوش مصنوعی از طریق منبع باز و علم باز هستیم.

huggingface.co

منتشر شده از طریق به سمت هوش مصنوعی