Phi-3.5をDatabricksで動かしてみる

こちらのPOSTを見かけて気になってました。

We released phi 3.5: mini+MoE+vision

A better mini model with multilingual support: https://t.co/f7avhBXHYn

A new MoE model:https://t.co/FxLILAqpEr

A new vision model supporting multiple images: https://t.co/rMkkpFc4cx

— Weizhu Chen (@WeizhuChen) August 20, 2024

Phi-3.5-mini-instruct

こちらのサンプルを使わせていただきます。

小さなGPUクラスターで動きました。

import torch
from transformers import AutoModelForCausalLM, AutoTokenizer, pipeline

torch.random.manual_seed(0)

model = AutoModelForCausalLM.from_pretrained(
    "microsoft/Phi-3-mini-4k-instruct", 
    device_map="cuda", 
    torch_dtype="auto", 
    trust_remote_code=True, 
)
tokenizer = AutoTokenizer.from_pretrained("microsoft/Phi-3-mini-4k-instruct")

messages = [
    {"role": "system", "content": "Your are a python developer."},
    {"role": "user", "content": "Help me generate a bubble algorithm"},
]

pipe = pipeline(
    "text-generation",
    model=model,
    tokenizer=tokenizer,
)

generation_args = {
    "max_new_tokens": 600,
    "return_full_text": False,
    "temperature": 0.3,
    "do_sample": False,
}

output = pipe(messages, **generation_args)
print(output[0]['generated_text'])

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
/databricks/python/lib/python3.11/site-packages/transformers/generation/configuration_utils.py:515: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.3` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.
  warnings.warn(
You are not running the flash-attention implementation, expect numerical differences.
 The Bubble Sort algorithm is a simple sorting algorithm that repeatedly steps through the list, compares adjacent elements, and swaps them if they are in the wrong order. The pass through the list is repeated until the list is sorted. Here's a Python implementation of the Bubble Sort algorithm:

```python
def bubble_sort(arr):
    n = len(arr)
    # Traverse through all array elements
    for i in range(n):
        # Last i elements are already in place
        for j in range(0, n-i-1):
            # Traverse the array from 0 to n-i-1
            # Swap if the element found is greater than the next element
            if arr[j] > arr[j+1]:
                arr[j], arr[j+1] = arr[j+1], arr[j]

# Example usage
arr = [64, 34, 25, 12, 22, 11, 90]
bubble_sort(arr)
print("Sorted array is:", arr)
```

This code defines a function `bubble_sort` that takes an array as input and sorts it using the Bubble Sort algorithm. The function uses two nested loops to compare adjacent elements and swap them if necessary. The outer loop runs `n` times, where `n` is the length of the array, and the inner loop runs `n-i-1` times, where `i` is the current iteration of the outer loop.

After defining the function, the code creates an example array `arr` with unsorted elements and calls the `bubble_sort` function to sort it. Finally, it prints the sorted array.

Keep in mind that Bubble Sort is not the most efficient sorting algorithm for large datasets, as its average and worst-case time complexity is O(n^2). For larger datasets, consider using more efficient algorithms like Quick Sort, Merge Sort, or Heap Sort.

日本語ではどうでしょう。

messages = [
    {"role": "system", "content": "あなたはPythonの開発者です。"},
    {"role": "user", "content": "バブルアルゴリズムの作成を助けてください。"},
]
output = pipe(messages, **generation_args)
print(output[0]['generated_text'])

/databricks/python/lib/python3.11/site-packages/transformers/generation/configuration_utils.py:515: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.3` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.
  warnings.warn(
 バブルソートは、比較と交換を繰り返してソートするアルゴリズムです。以下にPythonでのバブルソートの実装例を示します。


```python

def bubble_sort(arr):

    n = len(arr)

    for i in range(n):

        # 最初の未ソートの要素を探す

        for j in range(0, n-i-1):

            # 現在の要素と次の要素を比較し、順序が反転している場合

            if arr[j] > arr[j+1]:

                # 要素を交換

                arr[j], arr[j+1] = arr[j+1], arr[j]


# 例として、以下のリストをソートする

example_list = [64, 34, 25, 12, 22, 11, 90]

bubble_sort(example_list)

print("ソートされたリスト:", example_list)

```


このコードは、与えられたリストをバブルソートによってソートします。最初のループでは、リストの最後にある未ソート要素を探し、次のループではその要素と次の要素を比較し、順序が反転している場合は交換します。このプロセスはリストの最後から最初に繰り返され、最終的にはリストがソートされます。

そのままセルに貼り付けて動かしてみます。

def bubble_sort(arr):

    n = len(arr)

    for i in range(n):

        # 最初の未ソートの要素を探す

        for j in range(0, n-i-1):

            # 現在の要素と次の要素を比較し、順序が反転している場合

            if arr[j] > arr[j+1]:

                # 要素を交換

                arr[j], arr[j+1] = arr[j+1], arr[j]

# 例として、以下のリストをソートする

example_list = [64, 34, 25, 12, 22, 11, 90]

bubble_sort(example_list)

print("ソートされたリスト:", example_list)

ソートされたリスト: [11, 12, 22, 25, 34, 64, 90]

動きました！

Phi-3.5-vision-instruct

こちらはマルチモーダルモデルですね。

サンプルでは、以下のようなスライドの画像を要約してもらっています。

クラスターは先ほどより大きい、AシリーズのGPUインスタンスタイプを使います。使ったサイズでは20スライドはメモリーに乗り切らなかったので数を減らしています。

from PIL import Image 
import requests 
from transformers import AutoModelForCausalLM 
from transformers import AutoProcessor 

model_id = "microsoft/Phi-3.5-vision-instruct" 

# Note: set _attn_implementation='eager' if you don't have flash_attn installed
model = AutoModelForCausalLM.from_pretrained(
  model_id, 
  device_map="cuda", 
  trust_remote_code=True, 
  torch_dtype="auto", 
  _attn_implementation='flash_attention_2'    
)

# for best performance, use num_crops=4 for multi-frame, num_crops=16 for single-frame.
processor = AutoProcessor.from_pretrained(model_id, 
  trust_remote_code=True, 
  num_crops=4
) 

images = []
placeholder = ""

# Note: if OOM, you might consider reduce number of frames in this example.
for i in range(1,5):
    url = f"https://image.slidesharecdn.com/azureintroduction-191206101932/75/Introduction-to-Microsoft-Azure-Cloud-{i}-2048.jpg" 
    images.append(Image.open(requests.get(url, stream=True).raw))
    placeholder += f"<|image_{i}|>\n"

messages = [
    {"role": "user", "content": placeholder+"Summarize the deck of slides."},
]

prompt = processor.tokenizer.apply_chat_template(
  messages, 
  tokenize=False, 
  add_generation_prompt=True
)

inputs = processor(prompt, images, return_tensors="pt").to("cuda:0") 

generation_args = { 
    "max_new_tokens": 1000, 
    "temperature": 0.0, 
    "do_sample": False, 
} 

generate_ids = model.generate(**inputs, 
  eos_token_id=processor.tokenizer.eos_token_id, 
  **generation_args
)

# remove input tokens 
generate_ids = generate_ids[:, inputs['input_ids'].shape[1]:]
response = processor.batch_decode(generate_ids, 
  skip_special_tokens=True, 
  clean_up_tokenization_spaces=False)[0] 

print(response)

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
/databricks/python/lib/python3.11/site-packages/transformers/generation/configuration_utils.py:515: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.0` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.
  warnings.warn(
To encapsulate, the slides feature the following segments:

- Introduction to Azure: 
The presentation introduces Microsoft Azure, a cloud computing platform by Microsoft. It highlights Azure's three main types: Hyper-scale, Enterprise, and Hybrid. The presenter is Dinesh Kumar Wickramasinghe, a Senior Software Engineer from CMS Private Limited in Sri Lanka.

- Azure Overview: 
Azure is described as a continuously expanding set of cloud services that help organizations meet current and future business challenges. It offers the freedom to build, manage, and deploy applications on a massive global network using favorite tools and frameworks.

- Cloud Computing Services: 
The presentation outlines three types of cloud computing services provided by Azure: Infrastructure-as-a-Service (IaaS), Platform-as-a-Service (PaaS), and Software-as-a-Service (SaaS). Each service is represented by a unique icon and a brief description.

- Cloud Service Models: 
The presentation compares four cloud service models: On Premises, Infrastructure, Platform, and Software. Each model is represented by a column with different services listed under it, indicating the level of management and ownership.

メッセージを変えることで、日本語でも動作しました。

messages = [
    {"role": "user", "content": placeholder+"スライドのデックを要約してください。"},
]

Special tokens have been added in the vocabulary, make sure the associated word embeddings are fine-tuned or trained.
/databricks/python/lib/python3.11/site-packages/transformers/generation/configuration_utils.py:515: UserWarning: `do_sample` is set to `False`. However, `temperature` is set to `0.0` -- this flag is only used in sample-based generation modes. You should set `do_sample=True` or unset `temperature`.
  warnings.warn(

- スライド1: タイトルと著者の情報、Azureの概要、その種類（Hyper-scale、Enterprise、Hybrid）。
- スライド2: Azureの定義、その機能、そしてその範囲。
- スライド3: Azureのサービスの種類（IaaS、PaaS、SaaS）、それぞれの機能。
- スライド4: Azureのサービスの種類（On Premises、Infrastructure、Platform、Software）、それぞれの機能。

軽量なモデルがどんどん出てきて、どんどん使いやすくなっていますね。

はじめてのDatabricks

はじめてのDatabricks

Databricks無料トライアル

Databricks無料トライアル