LoginSignup
5
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

Space ROS (Robot Operating System)Advent Calendar 2024

Day 19
@mikami333(H Mikami)

【Space ROS】NASA開発コンペに出て、生成AIでローバーを動かしてみた

Last updated at Posted at 2024-12-21

1. はじめに

・NASAが管理しているOSS、SpaceROSの開発コンペがあったので、締め切りまで３週間だったができる限り取り組みました。
・ROS1の開発経験はありましたが、ROS2は初めてだったので、チュートリアルから勉強し直しました。
・細かい作業計画立案は生成AIに作らせて、人間のオペレータは、もっと他の作業に集中できるようにしたいと思います。

2.開発コンペ Summer Challengeについて

NASA主催のシミュレータ開発コンペが24年７月から９月に開催された。
公式ページ NASA Space ROS Sim Summer Sprint Challenge

Spacer ROS (Github Project)

3.開発環境構築

一番時間がかかりました。

・手元のLinuxPC・・・ビルド中に止まる。C++ とPythonとDocker Imageを同時並行でビルドするのに、PCスペックが必要。
・M1チップ対応のMacPC・・・ビルドはうまくいったが、Docker Imageが立ち上がらない。CPUのアーキテクチャがIntelのみ対応で、M1には対応していない。Rosettaシミュレータを使ってもダメだった。
・AWS EC2・・・Ubuntu Desktopをインストールするところで3D表示がうまくいかない。VPCserverという映像をクライアント先に表示するところで不具合があるよう。
・AWS Wokspaces・・・ようやく開発環境整った。Dockerもビルドも正常に作業できる。

4.開発成果物

ChatGPT APIを使って自然言語の指示でGazebo上のTurtleBot3を制御するを参考に作成しました。
実際のPullRequestに解説があるので記載。
作成したリポジトリはこちら

自然言語指示「10秒前進してから右に進んで」などを入れると、動作コードを書いて、動いてくれます。

動作前
スクリーンショット 2024-09-07 10.24.14.png

動作後
Screen shot of video 2024-09-07 10.24.53.png

重要なpythonコードとDockerfileを以下に記載します。

generate_src.py
import json
from openai import OpenAI
import subprocess
import os
import sys
from datetime import datetime

def load_config(file_path):
    try:
        with open(file_path, 'r') as file:
            return json.load(file)
    except FileNotFoundError:
        print(f"Error: {file_path} not found.")
        sys.exit(1)
    except json.JSONDecodeError:
        print(f"Error: {file_path} is not a valid JSON file.")
        sys.exit(1)

def load_prompt(file_path):
    try:
        with open(file_path, "r") as f:
            return f.read()
    except FileNotFoundError:
        print(f"Error: {file_path} not found.")
        sys.exit(1)

def get_chat_response(client, prompt, model):
    try:
        response = client.chat.completions.create(
            model=model,
            messages=[
                {"role": "user", "content": prompt},
            ]
        )
        return response.choices[0].message.content.strip()
    except Exception as e:
        print(f"Error getting response from GPT: {e}")
        sys.exit(1)

def remove_backticks(code_str):
    if code_str.startswith("```python"):
        code_str = code_str[len("```python"):].strip()
    if code_str.endswith("```"):
        code_str = code_str[:-len("```")].strip()
    return code_str

def generate_python_script(res):
    python_code = remove_backticks(res)
    current_time = datetime.now().strftime("%Y%m%d_%H%M")
    filename = f"gen_script_{current_time}.py"
    with open(filename, "w") as f:
        f.write(python_code)
    return filename

def run_python_script(filename):
    result = subprocess.run(["python3", filename], capture_output=True, text=True)
    if result.returncode != 0:
        print(f"Error executing script: {result.stderr}")
    else:
        print(f"Script output: {result.stdout}")

def main():
    config = load_config('config.json')
    client = OpenAI(api_key=config['openai_api_key'])

    pre_prompt = load_prompt("prompt.txt")
    print("Pre-prompt loaded successfully")

    print("Please input command like 'Go forward 10 seconds' and push enter-key")
    user_input = input("")
    prompt = pre_prompt + user_input

    res = get_chat_response(client, prompt, config['openai_model'])
    filename = generate_python_script(res)
    run_python_script(filename)

if __name__ == "__main__":
    main()

途中のprompt.txtには以下を記載しています。

prompt.txt
I need rclpy code to run a differential rover robot.
Publish /cmd_vel every 0.1 seconds.
Please only write Python code in your replies.
Reply by. import rclpy Please start with
QoS is not set

あと、OpenAI社のAPIキーやモデル名は、以下のようなJSONファイルを準備して読み込ませました。

dummy.json
{
  "openai_api_key": "sk-XXXXX",
  "openai_model": "gpt-4o-YYYYYY"
}

XXXやYYYは適切な値を入れてください。

gpt-4oが実際に生成したコードです。
ファイル名はあとから自分でつけました。

sample_code_by_gpt.py
#This code is generated by inputting "Rover goes forward 10m/s during 20 seconds"
#after 'python3 generate_src.py' 

import rclpy
from rclpy.node import Node
from geometry_msgs.msg import Twist
from time import sleep

class RoverController(Node):

    def __init__(self):
        super().__init__('rover_controller')
        self.publisher_ = self.create_publisher(Twist, '/cmd_vel', 10)
        self.timer = self.create_timer(0.1, self.publish_cmd_vel)
        self.start_time = self.get_clock().now()

    def publish_cmd_vel(self):
        current_time = self.get_clock().now()
        elapsed_time = (current_time - self.start_time).nanoseconds / 1e9

        if elapsed_time > 20:
            self.timer.cancel()  # Stop publishing after 20 seconds
            self.get_logger().info('Stopped sending velocity commands.')
            return

        cmd_vel = Twist()
        cmd_vel.linear.x = 10.0  # Forward speed in m/s
        cmd_vel.angular.z = 0.0  # No rotation

        self.publisher_.publish(cmd_vel)
        self.get_logger().info('Publishing: Linear X: %f, Angular Z: %f' %
                               (cmd_vel.linear.x, cmd_vel.angular.z))


def main(args=None):
    rclpy.init(args=args)
    rover_controller = RoverController()
    rclpy.spin(rover_controller)

    rover_controller.destroy_node()
    rclpy.shutdown()


if __name__ == '__main__':
    main()

Dockerfileのの記載は

# Copyright 2021 Open Source Robotics Foundation, Inc.
#
# Licensed under the Apache License, Version 2.0 (the "License");
# you may not use this file except in compliance with the License.
# You may obtain a copy of the License at
#
#     http://www.apache.org/licenses/LICENSE-2.0
#
# Unless required by applicable law or agreed to in writing, software
# distributed under the License is distributed on an "AS IS" BASIS,
# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
# See the License for the specific language governing permissions and
# limitations under the License.
#
# A Docker configuration script to build the Space ROS image.
#
# The script provides the following build arguments:
#
#   VCS_REF     - The git revision of the Space ROS source code (no default value).
#   VERSION     - The version of Space ROS (default: "preview")

#FROM openrobotics/moveit2:latest
FROM moveit/moveit2:humble-source

# Define arguments used in the metadata definition
ARG VCS_REF
ARG VERSION="preview"

# Specify the docker image metadata
LABEL org.label-schema.schema-version="1.0"
LABEL org.label-schema.name="Curiosity Rover"
LABEL org.label-schema.description="Curiosity rover demo on the Space ROS platform"
LABEL org.label-schema.vendor="Open Robotics"
LABEL org.label-schema.version=${VERSION}
LABEL org.label-schema.url="https://github.com/space-ros"
LABEL org.label-schema.vcs-url="https://github.com/space-ros/docker"
LABEL org.label-schema.vcs-ref=${VCS_REF}

# Define a few key variables
ENV DEMO_DIR=${HOME_DIR}/demos_ws
ENV IGNITION_VERSION fortress
ENV GZ_VERSION fortress
#ENV SPACEROS_DIR=/opt/spaceros

# Disable prompting during package installation
ARG DEBIAN_FRONTEND=noninteractive

# Get rosinstall_generator
# Using Docker BuildKit cache mounts for /var/cache/apt and /var/lib/apt ensures that
# the cache won't make it into the built image but will be maintained between steps.
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  sudo apt-get update -y && sudo apt-get install -y python3-rosinstall-generator

# TODO(anyone): remove demo-pkgs.txt, no packages left after exclusions
# Generate repos file for demo dependencies, excluding packages from Space ROS core.
# COPY --chown=${USERNAME}:${USERNAME} demo-pkgs.txt /tmp/
# COPY --chown=${USERNAME}:${USERNAME} excluded-pkgs.txt /tmp/
# RUN rosinstall_generator \
#   --rosdistro ${ROSDISTRO} \
#   --deps \
#   --exclude-path ${SPACEROS_DIR}/src \
#   --exclude-path ${MOVEIT2_DIR}/src \
#   --exclude $(cat /tmp/excluded-pkgs.txt) -- \
#   -- $(cat /tmp/demo-pkgs.txt) \
#   > /tmp/demo_generated_pkgs.repos

RUN mkdir -p ${DEMO_DIR}/src
WORKDIR ${DEMO_DIR}


# Install libmongoc for development
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  sudo apt-get install libmongoc-dev -y

# Compile mongo cxx driver https://mongocxx.org/mongocxx-v3/installation/linux/
RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  sudo apt-get install libssl-dev build-essential devscripts debian-keyring fakeroot debhelper cmake libboost-dev libsasl2-dev libicu-dev libzstd-dev doxygen wget -y

RUN wget https://github.com/mongodb/mongo-cxx-driver/releases/download/r3.6.7/mongo-cxx-driver-r3.6.7.tar.gz
RUN tar -xzf mongo-cxx-driver-r3.6.7.tar.gz
RUN cd mongo-cxx-driver-r3.6.7/build && cmake .. -DCMAKE_BUILD_TYPE=Release -DCMAKE_INSTALL_PREFIX=/usr/local && sudo cmake --build . --target EP_mnmlstc_core && cmake --build . && sudo cmake --build . --target install

# Get the source for the dependencies
# RUN vcs import src < /tmp/demo_generated_pkgs.repos
#RUN echo "SPACEROS_DIR is set to: ${SPACEROS_DIR}"
#RUN if [ -z "${SPACEROS_DIR}" ]; then echo "SPACEROS_DIR is empty"; else echo "SPACEROS_DIR is set to: ${SPACEROS_DIR}"; fi
ENV SPACEROS_DIR=/root/ws_moveit
ENV MOVEIT2_DIR=${SPACEROS_DIR}
#ENV ROSDISTRO=""

#RUN ls -l ${SPACEROS_DIR}  # この行を追加して、ディレクトリの内容を確認
RUN echo "SPACEROS_DIR is set to: ${SPACEROS_DIR}"

COPY --chown=${USERNAME}:${USERNAME} demo_manual_pkgs.repos /tmp/
RUN vcs import src < /tmp/demo_manual_pkgs.repos
RUN /bin/bash -c 'source "${SPACEROS_DIR}/install/setup.bash"' 

RUN --mount=type=cache,target=/var/cache/apt,sharing=locked \
  --mount=type=cache,target=/var/lib/apt,sharing=locked \
  sudo apt-get update -y 

RUN echo "MOVEIT2_DIR is set to: ${MOVEIT2_DIR}"

RUN /bin/bash -c 'source "${SPACEROS_DIR}/install/setup.bash"' 
RUN /bin/bash -c 'source "${MOVEIT2_DIR}/install/setup.bash"' 

RUN sudo apt-get update && sudo apt-get install -y curl gnupg2 lsb-release \
  && curl -s https://raw.githubusercontent.com/ros/rosdistro/master/ros.key | sudo apt-key add - \
  && sudo sh -c 'echo "deb http://packages.ros.org/ros2/ubuntu $(lsb_release -cs) main" > /etc/apt/sources.list.d/ros2-latest.list' \
  && sudo apt-get update

ENV ROSDISTRO=humble
RUN rosdep install --from-paths src --ignore-src -r -y --rosdistro ${ROSDISTRO}

# Build the demo
RUN /bin/bash -c 'source ${SPACEROS_DIR}/install/setup.bash && source ${MOVEIT2_DIR}/install/setup.bash \
  && colcon build --cmake-args -DCMAKE_BUILD_TYPE=Release'

ENV USERNAME=root 
 # ここでUSERNAMEを設定

# 確認用のデバッグコマンド
RUN echo "USERNAME is set to: $USERNAME"


# Add the user to the render group so that the user can access /dev/dri/renderD128
RUN sudo usermod -aG render $USERNAME

# Setup the entrypoint
COPY ./entrypoint.sh /
ENTRYPOINT ["/entrypoint.sh"]
CMD ["bash"]

4. 今後のアクション

・今回は、単純な動作はできたが、音声入力やより複雑な動作計画を立てれるように機能拡張したい。あと日本語対応も。

・nasa-jpl/rosaも気になる。Space ROSと繋げられるか試したいところ。

5
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
Sign upLogin
5
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?