はじめに

本記事では、コピーペーストで学べるAI入門数学基礎を扱います。
わからないところは生成AIに質問してください。

学べる内容まとめ

関数とグラフ（微分・線形・非線形）

一次関数・二次関数・指数関数・対数関数・サイン波の可視化
微分・勾配降下法（最急降下法・モーメンタム・Adam等）

教師あり学習（Supervised Learning）

分類：Logistic Regression, k-NN, SVM, 決定木, ランダムフォレスト, MLP, CNN, BERTなど
回帰：Linear, Ridge, Lasso, SVR, GBR, ニューラルネット

教師なし学習（Unsupervised Learning）

クラスタリング：k-means, DBSCAN, GMM, 階層的クラスタリング
次元削減：PCA, t-SNE, UMAP, Autoencoder

線形代数と確率統計

固有値・特異値分解・QR分解・行列のランク・最小二乗法
エントロピー・ジニ不純度・ベイズ推定・MCMC・条件付き確率
機械学習と連立方程式、60×60や3600次元空間の計算

時系列解析

トレンド・季節性・ノイズの合成と分解（FFT, ACF, PACF）
ローパス/ハイパス/バンドパスフィルタ
時系列データの前処理（補完・外れ値処理）

ニューラルネットワークと深層学習

ReLU, Sigmoidなどの活性化関数
ニューラルネットの自動微分（PyTorch）
CNN（画像分類）/ RNN（テキスト）/ Transformer（言語モデル）/ GAN / VAE

強化学習（Reinforcement Learning）

Q-Learning（表形式）
Policy Gradient（方策勾配法）
DQN（Deep Q-Network）

モデル評価・ハイパーパラメータ最適化

交差検証 / グリッドサーチ / ランダムサーチ / ROC曲線・AUC
混同行列 / MSE / F1スコア

関連リンクまとめ

Pythonコード

import numpy as np
import matplotlib.pyplot as plt

# -----------------------------
# データ生成 / Data generation
# -----------------------------
x = np.linspace(0.1, 10, 300)

# 各関数（機械学習テーマ付き） / ML-themed mathematical functions
y_linear = 2 * x + 3  # 線形回帰
y_quadratic = x**2 - 4 * x + 6  # 多項式回帰
y_exponential = np.exp(0.3 * x)  # 活性化関数（Softplusなど）
y_sqrt = np.sqrt(x)  # 特徴量スケーリング、距離
y_cuberoot = np.cbrt(x)  # 累乗根：ノイズの平滑化
y_log = np.log10(x)  # 特徴量の圧縮
y_ln = np.log(x)  # ロス関数や情報量（エントロピー）
y_sin = np.sin(x)  # 時系列信号（周期的パターン予測）

# -----------------------------
# プロット / Plotting with ML themes
# -----------------------------
plt.figure(figsize=(15, 12))

# 1次関数（線形回帰）
plt.subplot(4, 2, 1)
plt.plot(x, y_linear, label="Linear Regression: y = 2x + 3")
plt.title("Linear Function (Regression)")
plt.grid(True)
plt.legend()

# 2次関数（多項式回帰）
plt.subplot(4, 2, 2)
plt.plot(x, y_quadratic, label="Polynomial Regression: y = x² - 4x + 6", color='orange')
plt.title("Quadratic Function (Polynomial Regression)")
plt.grid(True)
plt.legend()

# 指数関数（活性化関数、Softmaxの一部など）
plt.subplot(4, 2, 3)
plt.plot(x, y_exponential, label="Activation: y = exp(0.3x)", color='green')
plt.title("Exponential Function (Activation Function)")
plt.grid(True)
plt.legend()

# 平方根・累乗根（前処理やスケーリング）
plt.subplot(4, 2, 4)
plt.plot(x, y_sqrt, label="Square Root: √x", color='purple')
plt.plot(x, y_cuberoot, label="Cube Root: ∛x", color='cyan')
plt.title("Root Functions (Feature Scaling)")
plt.grid(True)
plt.legend()

# 対数関数（情報理論、スケーリング）
plt.subplot(4, 2, 5)
plt.plot(x, y_log, label="Logarithmic (log₁₀)", color='brown')
plt.plot(x, y_ln, label="Natural Log (ln)", color='red')
plt.title("Logarithmic Functions (Information Compression)")
plt.grid(True)
plt.legend()

# サイン波（時系列予測モデル、LSTMなど）
plt.subplot(4, 2, 6)
plt.plot(x, y_sin, label="Sine Wave: sin(x)", color='blue')
plt.title("Sine Wave (Time Series Modeling)")
plt.grid(True)
plt.legend()

plt.suptitle("Machine Learning-Oriented Mathematical Functions", fontsize=16)
plt.tight_layout(rect=[0, 0, 1, 0.97])
plt.show()

import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LogisticRegression, LinearRegression
from sklearn.datasets import make_classification, make_regression
from sklearn.model_selection import train_test_split

# ----------------------------
# 分類データセットの作成 / Classification data
# ----------------------------
X_class, y_class = make_classification(n_samples=100, n_features=2, n_redundant=0, 
                                       n_clusters_per_class=1, random_state=42)
Xc_train, Xc_test, yc_train, yc_test = train_test_split(X_class, y_class, test_size=0.3, random_state=42)

# 分類モデルの訓練 / Train classifier
clf = LogisticRegression()
clf.fit(Xc_train, yc_train)

# ----------------------------
# 回帰データセットの作成 / Regression data
# ----------------------------
X_reg, y_reg = make_regression(n_samples=100, n_features=1, noise=15, random_state=42)
Xr_train, Xr_test, yr_train, yr_test = train_test_split(X_reg, y_reg, test_size=0.3, random_state=42)

# 回帰モデルの訓練 / Train regressor
reg = LinearRegression()
reg.fit(Xr_train, yr_train)

# ----------------------------
# プロット / Plotting
# ----------------------------
fig, axes = plt.subplots(1, 2, figsize=(12, 5))

# --- 分類の可視化 / Classification ---
axes[0].scatter(X_class[:, 0], X_class[:, 1], c=y_class, cmap='bwr', edgecolor='k')
x_vals = np.linspace(X_class[:, 0].min(), X_class[:, 0].max(), 100)
y_vals = -(clf.coef_[0][0] * x_vals + clf.intercept_[0]) / clf.coef_[0][1]
axes[0].plot(x_vals, y_vals, 'k--', label='Decision Boundary')
axes[0].set_title("Binary Classification (Logistic Regression)")
axes[0].legend()
axes[0].grid(True)

# --- 回帰の可視化 / Regression ---
axes[1].scatter(X_reg, y_reg, color='blue', label='Data')
axes[1].plot(X_reg, reg.predict(X_reg), color='red', label='Regression Line')
axes[1].set_title("Regression (Linear Regression)")
axes[1].legend()
axes[1].grid(True)

plt.tight_layout()
plt.show()

import numpy as np
import matplotlib.pyplot as plt

# ------------------------------
# ダミーデータを生成 / Generate overdetermined system
# ------------------------------
A = np.array([[1, 1], [2, 1], [3, 1]])
b = np.array([6, 8, 11])  # 観測値 / Observed values

# 最小二乗法による解 / Least squares solution
x_lstsq, residuals, rank, s = np.linalg.lstsq(A, b, rcond=None)
slope, intercept = x_lstsq

# 回帰直線を描画するためのデータ / Data for regression line
x_vals = np.linspace(0, 4, 100)
y_fit = slope * x_vals + intercept

# 観測点の x と予測値 / Residual analysis
x_obs = A[:, 0]
y_obs = b
y_pred = slope * x_obs + intercept
residuals_vec = y_obs - y_pred
R_squared = 1 - np.sum(residuals_vec**2) / np.sum((y_obs - np.mean(y_obs))**2)

# ------------------------------
# 可視化 / Plotting
# ------------------------------
plt.figure(figsize=(14, 5))

# 回帰直線と観測点
plt.subplot(1, 2, 1)
plt.scatter(x_obs, y_obs, color='blue', label='Observed Data')
plt.plot(x_vals, y_fit, color='red', label=f'Least Squares Fit: y = {slope:.2f}x + {intercept:.2f}')
for i in range(len(x_obs)):
    plt.plot([x_obs[i], x_obs[i]], [y_obs[i], y_pred[i]], 'k--', linewidth=0.8)  # 残差を線で表示
plt.xlabel('x')
plt.ylabel('y')
plt.title('Least Squares Regression Line with Residuals')
plt.grid(True)
plt.legend()

# 残差プロット
plt.subplot(1, 2, 2)
plt.hlines(0, x_obs.min(), x_obs.max(), colors='gray', linestyles='dashed')
plt.scatter(x_obs, residuals_vec, color='purple')
plt.plot(x_obs, residuals_vec, 'o--', color='purple')
plt.xlabel('x')
plt.ylabel('Residuals')
plt.title('Residual Plot')
plt.grid(True)

plt.tight_layout()
plt.show()

# ------------------------------
# 結果表示 / Output
# ------------------------------
{
    "最小二乗法の回帰式": f"y = {slope:.2f}x + {intercept:.2f}",
    "決定係数 R²": round(R_squared, 4),
    "残差": residuals_vec
}

import numpy as np
import matplotlib.pyplot as plt

# ------------------------------
# 入力と重みの定義 / Inputs and weights
# ------------------------------
inputs = np.random.rand(10)       # ランダムな入力（10個）/ 10 random inputs
weights = np.random.rand(10)      # ランダムな重み（10個）/ 10 random weights
bias = np.random.rand()           # バイアス項 / Bias term

# ------------------------------
# 総和計算 / Weighted sum
# ------------------------------
weighted_sum = np.dot(inputs, weights) + bias  # 入力×重み + バイアス / input * weight + bias

# ------------------------------
# 活性化関数（ReLU, Sigmoid）/ Activation functions
# ------------------------------
def relu(x):
    return np.maximum(0, x)

def sigmoid(x):
    return 1 / (1 + np.exp(-x))

relu_output = relu(weighted_sum)
sigmoid_output = sigmoid(weighted_sum)

# ------------------------------
# 出力表示 / Display results
# ------------------------------
print("入力 / Inputs:", inputs)
print("重み / Weights:", weights)
print("バイアス / Bias:", bias)
print("重み付き総和 / Weighted Sum:", weighted_sum)
print("ReLU出力 / ReLU Output:", relu_output)
print("Sigmoid出力 / Sigmoid Output:", sigmoid_output)

# ------------------------------
# 可視化 / Visualization
# ------------------------------
x_vals = np.linspace(-10, 10, 200)
relu_vals = relu(x_vals)
sigmoid_vals = sigmoid(x_vals)

plt.figure(figsize=(12, 5))

plt.subplot(1, 2, 1)
plt.plot(x_vals, relu_vals, label="ReLU")
plt.axvline(weighted_sum, color='r', linestyle='--', label="Input sum")
plt.title("ReLU Activation")
plt.grid(True)
plt.legend()

plt.subplot(1, 2, 2)
plt.plot(x_vals, sigmoid_vals, label="Sigmoid", color='green')
plt.axvline(weighted_sum, color='r', linestyle='--', label="Input sum")
plt.title("Sigmoid Activation")
plt.grid(True)
plt.legend()

plt.tight_layout()
plt.show()


import torch
import torch.nn.functional as F

# --- ステップ①：入力変数の定義（勾配追跡ON） ---
x = torch.tensor(2.0, requires_grad=True)  # 入力1
y = torch.tensor(3.0, requires_grad=True)  # 入力2

# --- ステップ②：演算の構成（順伝播） ---
# 線形部分（仮想的なニューロン）: z = x^2 * y + y^3
a = x**2
b = a * y
c = y**3
z = b + c

# --- ステップ③：活性化関数（ReLU）を通す ---
# Activation function (ReLU) applied to z
activated = F.relu(z)  # max(0, z)

# --- ステップ④：逆伝播前に勾配初期化 ---
x.grad = None
y.grad = None

# --- ステップ⑤：逆伝播（勾配計算） ---
activated.backward()

# --- ステップ⑥：結果の出力 ---
print(f"z（活性化前）= {z.item():.2f}")
print(f"ReLU(z) = {activated.item():.2f}")
print(f"∂ReLU(z)/∂x = {x.grad}")  # ReLUの勾配が0になる可能性あり
print(f"∂ReLU(z)/∂y = {y.grad}")

# 必要なライブラリの再インポート（状態リセットのため）
import matplotlib.pyplot as plt
import numpy as np

# ----------------------------
# データの生成 / Generate example data
# ----------------------------

# ベクトルと行列の基本 / Scalars, vectors, matrices
vector = np.array([2, 3])
matrix = np.array([[2, 1], [1, 3]])

# 線形変換 / Linear transformation
transformed_vector = matrix @ vector

# 固有値・固有ベクトル / Eigenvalues and eigenvectors
eigvals, eigvecs = np.linalg.eig(matrix)

# ----------------------------
# プロット / Plotting
# ----------------------------

fig, ax = plt.subplots(figsize=(7, 7))
ax.quiver(0, 0, vector[0], vector[1], angles='xy', scale_units='xy', scale=1, color='blue', label='Original Vector')
ax.quiver(0, 0, transformed_vector[0], transformed_vector[1], angles='xy', scale_units='xy', scale=1, color='red', label='Transformed Vector')
ax.quiver(0, 0, eigvecs[:,0][0], eigvecs[:,0][1], angles='xy', scale_units='xy', scale=1.5, color='green', label='Eigenvector 1')
ax.quiver(0, 0, eigvecs[:,1][0], eigvecs[:,1][1], angles='xy', scale_units='xy', scale=1.5, color='orange', label='Eigenvector 2')

ax.set_xlim(-1, 6)
ax.set_ylim(-1, 6)
ax.set_aspect('equal')
ax.axhline(0, color='gray', linewidth=0.5)
ax.axvline(0, color='gray', linewidth=0.5)
ax.grid(True)
ax.set_title("Linear Algebra Concepts for Machine Learning")
ax.legend()
plt.show()

import numpy as np
import matplotlib.pyplot as plt
import torch

# -------------------------------
# PyTorchで自動微分つきニューラルネットワークの学習実装
# -------------------------------

# 入力データと正解ラベル / Input and target
x = torch.tensor([0.6], requires_grad=False)
y_true = torch.tensor([1.0], requires_grad=False)

# 重みとバイアス（勾配追跡ON） / Weight and bias with gradient tracking
w = torch.randn(1, requires_grad=True)
b = torch.randn(1, requires_grad=True)

# 学習率とエポック数 / Learning rate and epochs
lr = 0.1
epochs = 50
losses = []

# シグモイド関数 / Sigmoid function
sigmoid = torch.nn.Sigmoid()

for epoch in range(epochs):
    # フォワードパス / Forward pass
    z = w * x + b
    a = sigmoid(z)
    loss = (a - y_true) ** 2  # 損失（二乗誤差）

    # バックプロパゲーション / Backward pass
    loss.backward()

    # パラメータの更新 / Update parameters
    with torch.no_grad():
        w -= lr * w.grad
        b -= lr * b.grad

    # 勾配をゼロにリセット / Reset gradients
    w.grad.zero_()
    b.grad.zero_()

    losses.append(loss.item())

# -------------------------------
# 学習結果の可視化 / Visualize loss over epochs
# -------------------------------
plt.figure(figsize=(6, 4))
plt.plot(losses, label='Loss')
plt.xlabel("Epoch")
plt.ylabel("Loss")
plt.title("Training Loss via Backpropagation")
plt.grid(True)
plt.legend()
plt.tight_layout()
plt.show()

# -------------------------------
# 最終結果 / Final weights and output
# -------------------------------
{
    "Final Weight": w.item(),
    "Final Bias": b.item(),
    "Final Prediction": a.item(),
    "Final Loss": loss.item()
}

import numpy as np
import matplotlib.pyplot as plt

# --- 入力データと正解値の設定 / Set input data and correct output (supervised learning) ---
x = np.array([1, 2, 3, 4, 5])     # 入力データ / Input
y_true = np.array([2.2, 4.1, 6.0, 8.1, 10.2])  # 正解値 / True values

# --- パラメータの初期化 / Initialize parameters ---
a = np.random.randn()  # 傾き (slope)
b = np.random.randn()  # 切片 (intercept)

# --- 学習率とエポック数 / Learning rate and epochs ---
lr = 0.01              # 学習率 / Learning rate
epochs = 1000          # 繰り返し回数 / Number of iterations

# --- 誤差記録用リスト / List to record loss ---
loss_history = []

# --- 損失関数（平均二乗誤差） / Loss function (Mean Squared Error) ---
def mse(y_pred, y_true):
    return np.mean((y_pred - y_true) ** 2)

# --- 学習ループ / Training loop ---
for epoch in range(epochs):
    # 予測値の計算 / Compute prediction
    y_pred = a * x + b

    # 誤差の計算 / Compute error
    error = y_pred - y_true

    # 損失の計算 / Compute loss
    loss = mse(y_pred, y_true)
    loss_history.append(loss)

    # --- 勾配の計算 / Compute gradients ---
    grad_a = 2 * np.mean(error * x)
    grad_b = 2 * np.mean(error)

    # --- パラメータの更新 / Update parameters ---
    a -= lr * grad_a
    b -= lr * grad_b

# --- 最終的なパラメータと誤差の出力 / Output final parameters and error ---
print(f"学習後のパラメータ / Trained parameters: a = {a:.4f}, b = {b:.4f}")
print(f"最小誤差 / Final loss: {loss_history[-1]:.4f}")

# --- 結果の可視化 / Plotting results ---
plt.figure(figsize=(10, 5))

# 予測線とデータ点 / Regression line and data points
plt.subplot(1, 2, 1)
plt.scatter(x, y_true, label="True Data")
plt.plot(x, a * x + b, color='red', label="Predicted Line")
plt.title("Prediction vs True")
plt.xlabel("x")
plt.ylabel("y")
plt.legend()

# 誤差の推移 / Loss transition
plt.subplot(1, 2, 2)
plt.plot(loss_history)
plt.title("Loss over Epochs")
plt.xlabel("Epoch")
plt.ylabel("MSE Loss")

plt.tight_layout()
plt.show()

import numpy as np
import matplotlib.pyplot as plt

# --- シグモイド関数の定義 / Sigmoid function ---
def sigmoid(z):
    return 1 / (1 + np.exp(-z))

# --- 損失関数（ロジスティック損失）/ Log-likelihood loss function ---
def loss_fn(y, y_pred):
    # クロスエントロピー / Cross-entropy
    return -np.mean(y * np.log(y_pred + 1e-8) + (1 - y) * np.log(1 - y_pred + 1e-8))

# --- 入力データ（2特徴）/ Input data with 2 features ---
# 例：クラスA=0, クラスB=1
X = np.array([
    [0.2, 0.1],   # A
    [0.4, 0.6],   # A
    [0.5, 0.9],   # B
    [0.9, 0.7],   # B
    [0.3, 0.2],   # A
    [0.8, 0.8]    # B
])

y = np.array([0, 0, 1, 1, 0, 1])  # 正解ラベル / Target labels (0 = A, 1 = B)

# --- パラメータの初期化 / Initialize weights and bias ---
np.random.seed(0)
w = np.random.randn(2)  # 重み / Weights
b = 0.0                 # バイアス / Bias

# --- 学習率とエポック数 / Learning rate and epochs ---
lr = 0.1
epochs = 1000
loss_history = []

# --- 学習ループ / Training loop ---
for epoch in range(epochs):
    z = np.dot(X, w) + b           # 総入力 / Linear combination
    y_pred = sigmoid(z)            # 予測確率 / Predicted probability

    loss = loss_fn(y, y_pred)      # 損失計算 / Compute loss
    loss_history.append(loss)

    # --- 勾配の計算 / Compute gradients ---
    dz = y_pred - y                # 誤差 / Error
    dw = np.dot(X.T, dz) / len(X) # 重みの勾配 / Gradient of weights
    db = np.sum(dz) / len(X)      # バイアスの勾配 / Gradient of bias

    # --- パラメータ更新 / Update parameters ---
    w -= lr * dw
    b -= lr * db

# --- 最終的な重みとバイアス / Final parameters ---
print(f"重み / Weights: {w}")
print(f"バイアス / Bias: {b}")
print(f"最終損失 / Final loss: {loss_history[-1]:.4f}")

# --- クラスAとBの確率を表示 / Display probabilities ---
for i in range(len(X)):
    prob_B = sigmoid(np.dot(X[i], w) + b)
    prob_A = 1 - prob_B
    print(f"入力{x} → A: {prob_A:.2f}, B: {prob_B:.2f}")

# --- 損失の推移を可視化 / Plot loss over epochs ---
plt.plot(loss_history)
plt.title("Loss (Negative Log Likelihood) over Epochs")
plt.xlabel("Epoch")
plt.ylabel("Loss")
plt.grid(True)
plt.show()

import numpy as np
import matplotlib.pyplot as plt

# --- ソフトマックス関数 / Softmax function ---
def softmax(z):
    exp_z = np.exp(z - np.max(z, axis=1, keepdims=True))  # 数値安定化 / for numerical stability
    return exp_z / np.sum(exp_z, axis=1, keepdims=True)

# --- クロスエントロピー損失（尤度関数） / Cross-entropy loss ---
def cross_entropy(y_true, y_pred):
    return -np.mean(np.sum(y_true * np.log(y_pred + 1e-8), axis=1))

# --- ワンホット変換 / One-hot encoding ---
def to_onehot(y, num_classes):
    onehot = np.zeros((len(y), num_classes))
    onehot[np.arange(len(y)), y] = 1
    return onehot

# --- データ作成 / Input data ---
X = np.array([
    [1.0, 2.0],  # class 0
    [1.5, 1.8],  # class 0
    [2.0, 3.0],  # class 0
    [3.0, 2.0],  # class 1
    [3.5, 1.5],  # class 1
    [4.0, 1.0],  # class 1
    [5.0, 3.0],  # class 2
    [5.5, 2.5],  # class 2
    [6.0, 2.0],  # class 2
])

y = np.array([0, 0, 0, 1, 1, 1, 2, 2, 2])  # 正解ラベル / Labels

num_classes = 3         # クラス数 / Number of classes
num_features = X.shape[1]
y_onehot = to_onehot(y, num_classes)  # ワンホットラベル

# --- パラメータ初期化 / Initialize weights and bias ---
np.random.seed(0)
W = np.random.randn(num_features, num_classes)  # 重み行列 / Weight matrix
b = np.zeros((1, num_classes))                  # バイアスベクトル / Bias vector

# --- ハイパーパラメータ / Hyperparameters ---
lr = 0.1
epochs = 1000
loss_history = []

# --- 学習ループ / Training loop ---
for epoch in range(epochs):
    z = np.dot(X, W) + b               # 総入力 / Linear combination
    y_pred = softmax(z)                # ソフトマックス出力 / Softmax output

    loss = cross_entropy(y_onehot, y_pred)  # 損失計算 / Compute loss
    loss_history.append(loss)

    # --- 勾配計算 / Compute gradients ---
    dz = y_pred - y_onehot
    dW = np.dot(X.T, dz) / len(X)
    db = np.sum(dz, axis=0, keepdims=True) / len(X)

    # --- パラメータ更新 / Update parameters ---
    W -= lr * dW
    b -= lr * db

# --- 結果出力 / Show results ---
print(f"学習後の重み / Trained weights:\n{W}")
print(f"学習後のバイアス / Trained bias:\n{b}")
print(f"最終損失 / Final loss: {loss_history[-1]:.4f}")

# --- 各サンプルのクラス確率と予測ラベル表示 / Show predicted probabilities and labels ---
print("\n各入力に対するクラス確率と予測：")
for i, xi in enumerate(X):
    probs = softmax(np.dot(xi, W) + b)
    predicted = np.argmax(probs)
    print(f"入力 {xi} → クラス確率: {probs.flatten()} → 予測: クラス{predicted}")

# --- 損失の推移グラフ / Plot loss over epochs ---
plt.plot(loss_history)
plt.title("Cross-Entropy Loss over Epochs")
plt.xlabel("Epoch")
plt.ylabel("Loss")
plt.grid(True)
plt.show()


# プログラム名: linear_system_least_squares_gradient.py
# 概要: 連立一次方程式を最小二乗法と勾配降下法の両方で解く
# Program to solve a linear system using least squares and gradient descent methods

import numpy as np
import matplotlib.pyplot as plt

# --- 問題設定: 連立一次方程式 Ax = b / Define linear system Ax = b ---
A = np.array([
    [2, 1],
    [1, 3],
    [0, 4],
    [5, 2]
])
b = np.array([7, 8, 9, 10])

print("Matrix A:\n", A)
print("Vector b:\n", b)

# --- Step 1: 最小二乗法による解法 / Solve by least squares ---
# 理論解: x = (A^T A)^(-1) A^T b
ATA_inv = np.linalg.inv(A.T @ A)
x_ls = ATA_inv @ (A.T @ b)

print("\n 最小二乗法による解 / Least Squares Solution:\n", x_ls)

# --- Step 2: 勾配降下法による近似解 / Solve by Gradient Descent ---
# 初期値設定 / Initialize
x_gd = np.zeros(A.shape[1])
learning_rate = 0.01
epochs = 500
loss_history = []

# 勾配降下法ループ / Gradient descent loop
for i in range(epochs):
    grad = 2 * A.T @ (A @ x_gd - b)    # 勾配を計算 / Compute gradient
    x_gd -= learning_rate * grad       # パラメータ更新 / Update x
    loss = np.linalg.norm(A @ x_gd - b)**2   # 損失(L2ノルムの二乗) / Compute loss
    loss_history.append(loss)

print("\n 勾配法による最終解 / Final Solution by Gradient Descent:\n", x_gd)

# --- Step 3: 損失関数の推移プロット / Plot loss curve ---
plt.figure(figsize=(8, 4))
plt.plot(loss_history)
plt.title('Loss Curve of Gradient Descent')
plt.xlabel('Epoch')
plt.ylabel('Loss (Squared Error)')
plt.grid(True)
plt.tight_layout()
plt.show()

# --- 最小二乗解と勾配法解の比較 / Compare Solutions ---
error = np.linalg.norm(x_ls - x_gd)
print(f"\n🆚 最小二乗法と勾配法の解の差 / Difference between solutions: {error:.6f}")

from sklearn.datasets import load_iris
from sklearn.tree import DecisionTreeClassifier, plot_tree
import matplotlib.pyplot as plt

# --- データの読み込み / Load classification data ---
iris = load_iris()
X, y = iris.data, iris.target

# --- ジニ不純度による分類木 / Decision tree (Gini) ---
clf_gini = DecisionTreeClassifier(criterion='gini', max_depth=3, random_state=0)
clf_gini.fit(X, y)

# --- エントロピーによる分類木 / Decision tree (Entropy) ---
clf_entropy = DecisionTreeClassifier(criterion='entropy', max_depth=3, random_state=0)
clf_entropy.fit(X, y)

# --- 分類木の可視化 / Plot decision tree ---
plt.figure(figsize=(12, 5))
plt.subplot(1, 2, 1)
plot_tree(clf_gini, feature_names=iris.feature_names, class_names=iris.target_names, filled=True)
plt.title("Decision Tree (Gini)")

plt.subplot(1, 2, 2)
plot_tree(clf_entropy, feature_names=iris.feature_names, class_names=iris.target_names, filled=True)
plt.title("Decision Tree (Entropy)")
plt.show()
from sklearn.datasets import make_regression
from sklearn.tree import DecisionTreeRegressor
import matplotlib.pyplot as plt
import numpy as np

# --- 回帰データ生成 / Generate regression data ---
X, y = make_regression(n_samples=100, n_features=1, noise=15, random_state=0)

# --- 決定木回帰器の訓練 / Train Decision Tree Regressor ---
reg = DecisionTreeRegressor(max_depth=3, random_state=0)
reg.fit(X, y)

# --- 予測と残差平方和の計算 / Compute predictions and RSS ---
y_pred = reg.predict(X)
rss = np.sum((y - y_pred) ** 2)
print(f"残差平方和 RSS = {rss:.2f}")

# --- 回帰木の可視化 / Plot regression tree ---
plt.figure(figsize=(6, 4))
plt.scatter(X, y, label='True Data', alpha=0.6)
plt.scatter(X, y_pred, color='red', label='Predicted', s=15)
plt.title("Decision Tree Regression")
plt.xlabel("x")
plt.ylabel("y")
plt.legend()
plt.grid(True)
plt.show()

import numpy as np
import matplotlib.pyplot as plt
import statsmodels.api as sm

# --- 媒介変数 t による円の生成 / Generate circle via parametric t ---
n = 200
r = 5
t = np.linspace(0, 2 * np.pi, n)

# --- 円の媒介変数表現 / Circle from parametric equations ---
x_true = r * np.cos(t)
y_true = r * np.sin(t)

# --- ノイズ追加 / Add noise ---
x_noisy = x_true + np.random.normal(0, 0.5, size=n)
y_noisy = y_true + np.random.normal(0, 0.5, size=n)

# --- 回帰モデルの作成（cos(t), sin(t)でx, yを表現）/ Fit regression ---
X_reg = sm.add_constant(np.column_stack([np.cos(t), np.sin(t)]))  # 定数項 + cos + sin

# x ~ cos(t), sin(t)
model_x = sm.OLS(x_noisy, X_reg).fit()
x_pred = model_x.predict(X_reg)

# y ~ cos(t), sin(t)
model_y = sm.OLS(y_noisy, X_reg).fit()
y_pred = model_y.predict(X_reg)

# --- 結果の可視化 / Visualization ---
plt.figure(figsize=(6, 6))
plt.plot(x_true, y_true, label='True Circle', linestyle='--', alpha=0.5)
plt.scatter(x_noisy, y_noisy, label='Noisy Observed Data', color='gray', s=10)
plt.plot(x_pred, y_pred, label='Regression Fit', color='red')
plt.title("Circle Regression via Mediator t")
plt.xlabel("x")
plt.ylabel("y")
plt.axis('equal')
plt.legend()
plt.grid(True)
plt.show()

# --- 回帰係数の出力 / Print model coefficients ---
print("x(t) 回帰係数 / Regression Coefficients for x:")
print(model_x.params)
print("\ny(t) 回帰係数 / Regression Coefficients for y:")
print(model_y.params)

import numpy as np
from sklearn.metrics.pairwise import cosine_similarity
from sklearn.feature_extraction.text import CountVectorizer

# ----------------------------
# サンプル文（簡単な単語ベース）/ Sample text documents
# ----------------------------
text1 = "machine learning artificial intelligence"
text2 = "deep learning neural network"

# CountVectorizerで単語ベクトル化 / Convert texts to bag-of-words vectors
vectorizer = CountVectorizer()
X = vectorizer.fit_transform([text1, text2]).toarray()

# 単語リスト表示 / Show vocabulary
vocab = vectorizer.get_feature_names_out()

# コサイン類似度の計算 / Compute cosine similarity
cos_sim = cosine_similarity([X[0]], [X[1]])[0][0]

# ----------------------------
# 出力 / Display
# ----------------------------
{
    "Vocabulary": vocab.tolist(),
    "Vector 1": X[0].tolist(),
    "Vector 2": X[1].tolist(),
    "Cosine Similarity": round(cos_sim, 4)
}

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm, bernoulli, binom

# ------------------------------
# 確率の種類と分布の例 / Probability types and distributions
# ------------------------------

# 1. ベルヌーイ分布（コイン投げ）/ Bernoulli
p = 0.6
bern_sample = bernoulli.rvs(p, size=1000)

# 2. 二項分布（n回投げて成功回数）/ Binomial
n, trials = 10, 1000
binom_sample = binom.rvs(n, p, size=trials)

# 3. 正規分布（確率密度）/ Normal Distribution
mu, sigma = 0, 1
x = np.linspace(-4, 4, 200)
pdf = norm.pdf(x, mu, sigma)

# ------------------------------
# 条件付き確率（AかつB / B）と同時確率（AかつB）例
# ------------------------------
P_A = 0.5
P_B_given_A = 0.7
P_A_and_B = P_A * P_B_given_A  # 同時確率 / Joint
P_B = 0.6
P_A_given_B = P_A_and_B / P_B  # 条件付き確率 / Conditional

# ------------------------------
# ベイズの定理による更新 / Bayes’ Theorem
# ------------------------------
# P(H|D) = P(D|H) * P(H) / P(D)
P_H = 0.01  # 病気の人の割合
P_D_given_H = 0.9  # 陽性になる確率（感度）
P_D_given_notH = 0.05  # 偽陽性率
P_notH = 1 - P_H

P_D = P_D_given_H * P_H + P_D_given_notH * P_notH
P_H_given_D = (P_D_given_H * P_H) / P_D  # 陽性だったときに病気である確率

# ------------------------------
# 期待値と分散の計算 / Expected value and variance
# ------------------------------
expected_value = np.mean(binom_sample)
variance = np.var(binom_sample)

# ------------------------------
# 最尤推定 / Maximum Likelihood Estimation
# ------------------------------
# 観測データからpの最尤推定（コインの裏表）
obs = np.random.binomial(1, 0.65, 100)
p_mle = np.mean(obs)

# ------------------------------
# プロット表示 / Plot
# ------------------------------
plt.figure(figsize=(14, 6))

# 正規分布
plt.subplot(1, 2, 1)
plt.plot(x, pdf, label='Normal Distribution')
plt.title('Normal Distribution PDF')
plt.xlabel('x')
plt.ylabel('Probability Density')
plt.grid(True)
plt.legend()

# ベルヌーイ + 二項ヒストグラム
plt.subplot(1, 2, 2)
plt.hist(binom_sample, bins=np.arange(n+2)-0.5, alpha=0.7, label='Binomial')
plt.title('Binomial Distribution (n=10, p=0.6)')
plt.xlabel('Success Count')
plt.ylabel('Frequency')
plt.grid(True)
plt.legend()

plt.tight_layout()
plt.show()

# ------------------------------
# 結果の出力 / Output
# ------------------------------
{
    "P(A and B)": round(P_A_and_B, 4),
    "P(A | B)": round(P_A_given_B, 4),
    "P(H | D) by Bayes": round(P_H_given_D, 4),
    "Expected Value (Binomial)": round(expected_value, 2),
    "Variance (Binomial)": round(variance, 2),
    "MLE Estimate of p (Bernoulli)": round(p_mle, 4)
}

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import entropy
from sklearn.metrics import mean_squared_error

# -------------------------------
# MSE（理論・実践） / Mean Squared Error
# -------------------------------
y_true = np.array([3, -0.5, 2, 7])
y_pred = np.array([2.5, 0.0, 2, 8])
mse = mean_squared_error(y_true, y_pred)

# -------------------------------
# 最小二乗法（正規方程式）
# -------------------------------
X = np.array([[1, 1], [1, 2], [1, 3]])
y = np.array([1, 2, 3])
theta_normal = np.linalg.inv(X.T @ X) @ X.T @ y  # 正規方程式

# -------------------------------
# 最小二乗法（勾配降下法）
# -------------------------------
theta_gd = np.random.randn(2)
alpha = 0.1
epochs = 100
losses = []

for _ in range(epochs):
    gradient = 2 * X.T @ (X @ theta_gd - y) / len(y)
    theta_gd -= alpha * gradient
    loss = np.mean((X @ theta_gd - y)**2)
    losses.append(loss)

# -------------------------------
# 情報エントロピー（理論・実践）
# -------------------------------
probs = np.array([0.1, 0.2, 0.3, 0.4])
entropy_val = entropy(probs, base=2)

# -------------------------------
# ジニ不純度（理論・実践）
# -------------------------------
def gini(p):
    return 1 - np.sum(p**2)

gini_val = gini(probs)

# -------------------------------
# 様々な距離（ユークリッド・マンハッタン・コサイン）
# -------------------------------
v1 = np.array([1, 2, 3])
v2 = np.array([4, 5, 6])
euclidean = np.linalg.norm(v1 - v2)
manhattan = np.sum(np.abs(v1 - v2))
cosine = np.dot(v1, v2) / (np.linalg.norm(v1) * np.linalg.norm(v2))

# -------------------------------
# グラフ：勾配降下の損失変化 / GD loss curve
# -------------------------------
plt.figure(figsize=(6, 4))
plt.plot(losses, label="Gradient Descent Loss")
plt.xlabel("Epoch")
plt.ylabel("Loss")
plt.title("MSE Loss during Gradient Descent")
plt.grid(True)
plt.legend()
plt.tight_layout()
plt.show()

# -------------------------------
# 結果表示 / Output
# -------------------------------
{
    "MSE": round(mse, 4),
    "Theta (Normal Equation)": theta_normal.tolist(),
    "Theta (Gradient Descent)": theta_gd.tolist(),
    "Entropy (bits)": round(entropy_val, 4),
    "Gini Impurity": round(gini_val, 4),
    "Euclidean Distance": round(euclidean, 4),
    "Manhattan Distance": round(manhattan, 4),
    "Cosine Similarity": round(cosine, 4)
}

import numpy as np
import matplotlib.pyplot as plt

# -----------------------------
# 勾配ベクトルと勾配降下法の可視化例（2変数関数）
# -----------------------------

# 目的関数 f(x, y) = (x-2)^2 + (y-3)^2（凸関数）
def f(x, y):
    return (x - 2)**2 + (y - 3)**2

# 勾配（偏微分）/ Gradient vector
def grad_f(x, y):
    df_dx = 2 * (x - 2)
    df_dy = 2 * (y - 3)
    return np.array([df_dx, df_dy])

# 初期点 / Starting point
point = np.array([0.0, 0.0])
alpha = 0.1
history = [point.copy()]

# 勾配降下法 / Gradient descent iterations
for _ in range(20):
    grad = grad_f(point[0], point[1])
    point -= alpha * grad
    history.append(point.copy())

history = np.array(history)

# -----------------------------
# 描画（等高線 + 勾配ベクトル + 降下経路）
# -----------------------------
x_vals = np.linspace(-1, 4, 100)
y_vals = np.linspace(-1, 6, 100)
X, Y = np.meshgrid(x_vals, y_vals)
Z = f(X, Y)

plt.figure(figsize=(8, 6))
# 等高線表示
contours = plt.contour(X, Y, Z, levels=20, cmap='viridis')
plt.clabel(contours, inline=True, fontsize=8)

# 勾配ベクトル表示
for i in range(0, 100, 10):
    for j in range(0, 100, 10):
        gx, gy = grad_f(x_vals[i], y_vals[j])
        plt.arrow(x_vals[i], y_vals[j], -0.1 * gx, -0.1 * gy, 
                  head_width=0.1, color='gray', alpha=0.5)

# 降下経路表示
plt.plot(history[:, 0], history[:, 1], marker='o', color='red', label='Gradient Descent Path')
plt.title("Gradient Vectors and Descent Trajectory")
plt.xlabel("x")
plt.ylabel("y")
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()

import numpy as np
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
from sklearn.metrics.pairwise import euclidean_distances, cosine_similarity

# -------------------------------
# 高次元ベクトルの生成と距離計算（60x60=3600次元）
# -------------------------------
np.random.seed(42)
vec1 = np.random.rand(3600)
vec2 = np.random.rand(3600)

euclidean = np.linalg.norm(vec1 - vec2)
cosine = np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2))

# -------------------------------
# ディープラーニングと連立方程式：Ax = b
# -------------------------------
W = np.random.randn(60, 60)
x_true = np.random.randn(60)
b = W @ x_true
x_solved = np.linalg.lstsq(W, b, rcond=None)[0]
mse = np.mean((x_solved - x_true) ** 2)
error_vec = x_solved - x_true

# -------------------------------
# PCAによる次元削減（3600次元 → 2次元）
# -------------------------------
samples = np.random.rand(100, 3600)
pca = PCA(n_components=2)
samples_2d = pca.fit_transform(samples)

plt.figure(figsize=(6, 5))
plt.scatter(samples_2d[:, 0], samples_2d[:, 1], c='blue', alpha=0.6)
plt.title("PCA: 3600-D vectors reduced to 2D")
plt.xlabel("PC1")
plt.ylabel("PC2")
plt.grid(True)
plt.tight_layout()
plt.show()

# -------------------------------
# 誤差の分布をプロット
# -------------------------------
plt.figure(figsize=(6, 4))
plt.hist(error_vec, bins=20, color='salmon', edgecolor='black')
plt.title("Error Distribution: x_solved - x_true")
plt.xlabel("Error Value")
plt.ylabel("Frequency")
plt.grid(True)
plt.tight_layout()
plt.show()

# -------------------------------
# ミニバッチ線形更新（ミニ連立の近似解）
# -------------------------------
x_batch = np.random.randn(5, 60)
b_batch = x_batch @ W.T
solved_batch = []

for b_i in b_batch:
    sol = np.linalg.lstsq(W, b_i, rcond=None)[0]
    solved_batch.append(sol)

solved_batch = np.array(solved_batch)
batch_mse = np.mean((solved_batch - x_batch) ** 2)

# -------------------------------
# 出力表示
# -------------------------------
print("=== 3600次元ベクトルの距離 ===")
print(f"Euclidean Distance: {euclidean:.4f}")
print(f"Cosine Similarity: {cosine:.4f}")

print("\n=== 単一連立方程式 Ax = b ===")
print(f"Reconstructed x MSE: {mse:.6f}")

print("\n=== ミニバッチ連立方程式 ===")
print(f"Mini-batch MSE (5 samples): {batch_mse:.6f}")

import numpy as np
import matplotlib.pyplot as plt
from sklearn.decomposition import PCA
from sklearn.mixture import GaussianMixture
from scipy.linalg import svd

# ------------------------------
# 線形代数 - 行列の積と固有値 / Linear Algebra - Matrix multiplication and eigenvalues
# ------------------------------
# 行列Aとベクトルbの定義 / Define matrix A and vector b
A = np.array([[4, 1], [2, 3]])
b = np.array([1, 2])

# 行列の積 / Matrix multiplication
result = np.dot(A, b)

# 固有値・固有ベクトル / Eigenvalues and Eigenvectors
eigvals, eigvecs = np.linalg.eig(A)

print("行列Aの積:", result)
print("行列Aの固有値:", eigvals)
print("行列Aの固有ベクトル:", eigvecs)

# ------------------------------
# 特異値分解（SVD）/ Singular Value Decomposition (SVD)
# ------------------------------
# 特異値分解 / Perform SVD
U, S, Vt = svd(A)
print("\nSVD - U:", U)
print("SVD - S:", S)
print("SVD - Vt:", Vt)

# ------------------------------
# PCAによる次元削減 / PCA for dimensionality reduction
# ------------------------------
# ランダムなデータサンプルの生成 / Generate random data samples
data = np.random.rand(10, 3)

# PCAを実行 / Perform PCA
pca = PCA(n_components=2)
pca_result = pca.fit_transform(data)

print("\nPCAで2次元に削減したデータ:")
print(pca_result)

# ------------------------------
# 微積分学 - バックプロパゲーションと勾配降下法 / Backpropagation and Gradient Descent
# ------------------------------
# 損失関数（例: 二乗誤差）/ Loss function (e.g., Mean Squared Error)
def loss_function(w, x, y):
    return np.mean((x.dot(w) - y)**2)

# 勾配降下法 / Gradient Descent
def gradient_descent(x, y, learning_rate=0.01, epochs=100):
    w = np.random.randn(x.shape[1])
    loss_history = []

    for epoch in range(epochs):
        gradient = 2 * x.T.dot(x.dot(w) - y) / len(y)  # 勾配 / Gradient
        w -= learning_rate * gradient  # パラメータ更新 / Update parameters
        loss_history.append(loss_function(w, x, y))  # 損失記録 / Record loss

    return w, loss_history

# サンプルデータ / Sample data
x_data = np.random.rand(100, 3)
y_data = np.random.rand(100)

# 勾配降下法の実行 / Perform Gradient Descent
w_optimal, loss_history = gradient_descent(x_data, y_data)

print("\n最適な重み:", w_optimal)

# 損失の推移をプロット / Plot loss history
plt.plot(loss_history)
plt.title("Gradient Descent Loss Over Epochs")
plt.xlabel("Epochs")
plt.ylabel("Loss")
plt.grid(True)
plt.show()

# ------------------------------
# 確率論 - 条件付き確率 / Probability - Conditional Probability
# ------------------------------
# 確率P(A)と条件付き確率P(B|A) / Probability P(A) and Conditional Probability P(B|A)
P_A = 0.5
P_B_given_A = 0.7
P_A_and_B = P_A * P_B_given_A  # 同時確率 / Joint Probability
P_B = 0.6
P_A_given_B = P_A_and_B / P_B  # 条件付き確率 / Conditional Probability

print("\n条件付き確率 P(A|B):", P_A_given_B)

# ------------------------------
# 期待値最大化（EMアルゴリズム） / Expectation-Maximization (EM Algorithm)
# ------------------------------
# 正規分布に基づいた混合モデル / Gaussian Mixture Model based on Normal Distribution
gmm = GaussianMixture(n_components=2, random_state=42)
data_for_gmm = np.random.rand(100, 1)  # サンプルデータ / Sample data

# EMアルゴリズムで学習 / Learn using EM algorithm
gmm.fit(data_for_gmm)

# 各サンプルのクラスター予測 / Cluster predictions for each sample
cluster_preds = gmm.predict(data_for_gmm)

# 結果表示 / Display results
print("\nEMアルゴリズムの学習結果")
print("混合係数:", gmm.weights_)
print("各成分の平均:", gmm.means_)
print("各成分の分散:", gmm.covariances_)

import torch
import torch.nn as nn
import torch.optim as optim
from torch.utils.data import DataLoader, TensorDataset
import numpy as np
import matplotlib.pyplot as plt

# ------------------------------
# 1. Transformer
# ------------------------------
# Transformerモデルの定義 / Define the Transformer Model
class TransformerModel(nn.Module):
    def __init__(self, d_model, nhead, num_encoder_layers, num_decoder_layers, vocab_size):
        super(TransformerModel, self).__init__()
        # 埋め込み層 / Embedding layer
        self.embedding = nn.Embedding(vocab_size, d_model)
        # Transformer層 / Transformer layers
        self.transformer = nn.Transformer(d_model, nhead, num_encoder_layers, num_decoder_layers)
        # 出力層 / Output layer
        self.fc_out = nn.Linear(d_model, vocab_size)
        
    def forward(self, src, tgt):
        # 入力データを埋め込む / Embed input data
        embedded_src = self.embedding(src)
        embedded_tgt = self.embedding(tgt)
        # Transformerを通す / Pass through Transformer
        output = self.transformer(embedded_src, embedded_tgt)
        # 出力を計算 / Compute output
        output = self.fc_out(output)
        return output

# Simple dataset for Transformer model
# 例としてのデータセット / Example dataset for Transformer model
src = torch.randint(0, 10, (5, 32))  # 5語のシーケンス、バッチサイズ32 / Sequence of 5 words, batch size 32
tgt = torch.randint(0, 10, (5, 32))  # 同じくターゲットのシーケンス / Target sequence
model = TransformerModel(d_model=64, nhead=4, num_encoder_layers=2, num_decoder_layers=2, vocab_size=10)
output = model(src, tgt)
print("Transformer 出力の形状 / Transformer Output Shape:", output.shape)

# ------------------------------
# 2. BERT
# ------------------------------
from transformers import BertTokenizer, BertForMaskedLM

# BERTのモデルとトークナイザーをロード / Load BERT model and tokenizer
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model_bert = BertForMaskedLM.from_pretrained('bert-base-uncased')

# 入力テキスト / Input text
text = "Machine learning is [MASK] awesome."

# テキストをトークン化 / Tokenize text
inputs = tokenizer(text, return_tensors="pt")

# BERTの出力 / BERT output
with torch.no_grad():
    outputs = model_bert(**inputs)

# マスクされたトークンの予測 / Prediction of masked token
logits = outputs.logits
predicted_token_id = torch.argmax(logits[0, 5]).item()  # マスクされたトークンの位置 / Position of masked token
predicted_token = tokenizer.decode(predicted_token_id)
print(f"BERTの予測: {predicted_token} / BERT Prediction: {predicted_token}")

# ------------------------------
# 3. GAN (Generative Adversarial Network)
# ------------------------------
# ジェネレーターモデル / Generator Model
class Generator(nn.Module):
    def __init__(self):
        super(Generator, self).__init__()
        self.fc = nn.Sequential(
            nn.Linear(100, 256),
            nn.ReLU(True),
            nn.Linear(256, 512),
            nn.ReLU(True),
            nn.Linear(512, 784),
            nn.Tanh()
        )

    def forward(self, x):
        return self.fc(x)

# 判別モデル / Discriminator Model
class Discriminator(nn.Module):
    def __init__(self):
        super(Discriminator, self).__init__()
        self.fc = nn.Sequential(
            nn.Linear(784, 512),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Linear(512, 256),
            nn.LeakyReLU(0.2, inplace=True),
            nn.Linear(256, 1),
            nn.Sigmoid()
        )

    def forward(self, x):
        return self.fc(x)

# ジェネレータと判別器を初期化 / Initialize generator and discriminator
generator = Generator()
discriminator = Discriminator()

# ノイズを生成 / Generate noise for the generator
noise = torch.randn(16, 100)  # 16個のサンプル、ノイズベクトルの次元は100 / 16 samples, noise vector dimension = 100

# ジェネレータの出力 / Output from the generator
generated_images = generator(noise)
print("GAN出力画像の形状 / GAN Output Image Shape:", generated_images.shape)

# ------------------------------
# 4. VAE (Variational Autoencoder)
# ------------------------------
class VAE(nn.Module):
    def __init__(self, latent_dim):
        super(VAE, self).__init__()
        # エンコーダー / Encoder
        self.fc1 = nn.Linear(784, 400)
        self.fc21 = nn.Linear(400, latent_dim)  # 平均 / Mean
        self.fc22 = nn.Linear(400, latent_dim)  # 対数分散 / Log variance
        
        # デコーダー / Decoder
        self.fc3 = nn.Linear(latent_dim, 400)
        self.fc4 = nn.Linear(400, 784)

    def encode(self, x):
        h1 = torch.relu(self.fc1(x))
        return self.fc21(h1), self.fc22(h1)  # 平均と分散 / Mean and variance

    def reparameterize(self, mu, logvar):
        std = torch.exp(0.5*logvar)
        eps = torch.randn_like(std)
        return mu + eps*std  # 再パラメータ化 / Reparameterization trick

    def decode(self, z):
        h3 = torch.relu(self.fc3(z))
        return torch.sigmoid(self.fc4(h3))  # 出力は784次元の画像 / Output is 784-dimensional image

    def forward(self, x):
        mu, logvar = self.encode(x.view(-1, 784))
        z = self.reparameterize(mu, logvar)
        return self.decode(z), mu, logvar

# VAEを初期化 / Initialize VAE
vae = VAE(latent_dim=20)

# ランダムな画像データ / Random image data
image_data = torch.randn(16, 784)

# VAEの出力画像 / Output image from VAE
reconstructed_images, mu, logvar = vae(image_data)
print("VAE再構成画像の形状 / VAE Reconstructed Image Shape:", reconstructed_images.shape)

# ------------------------------
# 可視化 / Visualization
# ------------------------------
# GAN出力の可視化 / Visualizing GAN Output
plt.imshow(generated_images[0].detach().numpy().reshape(28, 28), cmap='gray')
plt.title("GAN Generated Image")
plt.show()

# VAE再構成画像の可視化 / Visualizing VAE Reconstructed Image
plt.imshow(reconstructed_images[0].detach().numpy().reshape(28, 28), cmap='gray')
plt.title("VAE Reconstructed Image")
plt.show()

import numpy as np
import torch
import torch.nn as nn
import torch.optim as optim
import random
import matplotlib.pyplot as plt
from collections import deque

# ------------------------------
# 1. Q学習 (Q-Learning)
# ------------------------------
class QLearningAgent:
    def __init__(self, action_space, learning_rate=0.1, discount_factor=0.9, epsilon=0.1):
        self.action_space = action_space
        self.learning_rate = learning_rate
        self.discount_factor = discount_factor
        self.epsilon = epsilon
        self.q_table = np.zeros(action_space)  # Qテーブルの初期化 / Initialize Q-table

    def act(self, state):
        if random.uniform(0, 1) < self.epsilon:  # 探索 / Exploration
            return random.randint(0, self.action_space - 1)
        else:  # 活用 / Exploitation
            return np.argmax(self.q_table)  # 最もQ値が高いアクションを選択 / Choose the action with highest Q-value

    def learn(self, state, action, reward, next_state):
        best_next_action = np.argmax(self.q_table)  # 次の状態での最良アクション / Best action for next state
        td_target = reward + self.discount_factor * self.q_table[best_next_action]
        self.q_table[action] += self.learning_rate * (td_target - self.q_table[action])  # Q値の更新 / Update Q-value

# 簡単な環境 (行動スペース 3) / Simple environment (3 action space)
env_size = 5
actions = 3
agent = QLearningAgent(actions)

# Q学習の実行 / Run Q-learning
episodes = 100
for episode in range(episodes):
    state = random.randint(0, env_size - 1)
    action = agent.act(state)
    reward = random.choice([0, 1])  # 報酬はランダム / Reward is random
    next_state = (state + 1) % env_size  # 状態遷移 / State transition
    agent.learn(state, action, reward, next_state)

# Qテーブルの表示 / Display Q-table
print("Q-Table after Q-learning:", agent.q_table)


# ------------------------------
# 2. ポリシー勾配法 (Policy Gradient)
# ------------------------------
class PolicyNetwork(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(PolicyNetwork, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.fc2 = nn.Linear(hidden_size, output_size)
    
    def forward(self, x):
        x = torch.relu(self.fc1(x))
        return torch.softmax(self.fc2(x), dim=-1)

class PolicyGradientAgent:
    def __init__(self, state_size, action_size):
        self.policy_net = PolicyNetwork(state_size, 16, action_size)
        self.optimizer = optim.Adam(self.policy_net.parameters(), lr=0.01)
    
    def act(self, state):
        state_tensor = torch.tensor(state, dtype=torch.float32)
        action_probs = self.policy_net(state_tensor)
        action = torch.multinomial(action_probs, 1).item()  # 確率に基づいてアクションを選択 / Choose action based on probabilities
        return action, action_probs[action]
    
    def learn(self, log_probs, reward):
        loss = -log_probs * reward  # ロスを計算 / Compute loss
        self.optimizer.zero_grad()
        loss.backward()
        self.optimizer.step()

# 簡単な環境 (状態空間 1, 行動空間 3) / Simple environment (1 state space, 3 action space)
agent_pg = PolicyGradientAgent(1, 3)

# ポリシー勾配法の実行 / Run Policy Gradient
states = [random.randint(0, 10)]
actions = []
log_probs = []
rewards = []

for _ in range(100):
    state = random.choice(states)
    action, log_prob = agent_pg.act([state])
    reward = random.choice([0, 1])
    
    actions.append(action)
    log_probs.append(log_prob)
    rewards.append(reward)

    # 学習 / Learn
    agent_pg.learn(log_probs[-1], reward)

print("ポリシー勾配法によるアクションの選択:", actions)


# ------------------------------
# 3. 深層Qネットワーク (Deep Q-Network, DQN)
# ------------------------------
class QNetwork(nn.Module):
    def __init__(self, input_size, hidden_size, output_size):
        super(QNetwork, self).__init__()
        self.fc1 = nn.Linear(input_size, hidden_size)
        self.fc2 = nn.Linear(hidden_size, output_size)

    def forward(self, x):
        x = torch.relu(self.fc1(x))
        return self.fc2(x)

class DQNAgent:
    def __init__(self, state_size, action_size, epsilon=0.1, gamma=0.9, lr=0.001):
        self.action_size = action_size
        self.epsilon = epsilon
        self.gamma = gamma
        self.q_network = QNetwork(state_size, 64, action_size)
        self.target_network = QNetwork(state_size, 64, action_size)
        self.optimizer = optim.Adam(self.q_network.parameters(), lr=lr)
        self.memory = deque(maxlen=2000)
    
    def act(self, state):
        if random.uniform(0, 1) < self.epsilon:
            return random.choice(range(self.action_size))
        else:
            state_tensor = torch.tensor(state, dtype=torch.float32)
            q_values = self.q_network(state_tensor)
            return torch.argmax(q_values).item()
    
    def store_experience(self, state, action, reward, next_state, done):
        self.memory.append((state, action, reward, next_state, done))
    
    def replay(self, batch_size=32):
        if len(self.memory) < batch_size:
            return

        minibatch = random.sample(self.memory, batch_size)

        for state, action, reward, next_state, done in minibatch:
            state_tensor = torch.tensor(state, dtype=torch.float32)
            next_state_tensor = torch.tensor(next_state, dtype=torch.float32)
            reward_tensor = torch.tensor(reward, dtype=torch.float32)
            done_tensor = torch.tensor(done, dtype=torch.float32)
            
            q_values = self.q_network(state_tensor)
            next_q_values = self.target_network(next_state_tensor)
            
            target = reward_tensor + (self.gamma * torch.max(next_q_values) * (1 - done_tensor))
            loss = nn.MSELoss()(q_values[action], target)

            self.optimizer.zero_grad()
            loss.backward()
            self.optimizer.step()

# 環境とエージェント / Environment and Agent
state_size = 4
action_size = 2
agent_dqn = DQNAgent(state_size, action_size)

# 深層Qネットワークの実行 / Run DQN
for e in range(100):
    state = np.random.rand(state_size)  # 状態 / State
    action = agent_dqn.act(state)  # アクション / Action
    reward = random.choice([0, 1])  # 報酬 / Reward
    next_state = np.random.rand(state_size)  # 次の状態 / Next state
    done = random.choice([True, False])  # 終了状態 / Done

    # 経験を保存 / Store experience
    agent_dqn.store_experience(state, action, reward, next_state, done)

    # 経験を再生 / Replay experience
    agent_dqn.replay(batch_size=32)

print("DQNエージェントの学習完了 / DQN agent learning completed.")

import numpy as np
import matplotlib.pyplot as plt
import pandas as pd
from scipy.signal import butter, filtfilt
from statsmodels.graphics.tsaplots import plot_acf, plot_pacf
from scipy.fft import fft, ifft

# ------------------------------
# 1. 時系列データとは？
# ------------------------------
# 時系列データの定義と特徴 / Definition and Characteristics of Time Series Data
# 時系列データは、時間の経過とともに収集されたデータのことです。  
# 例えば、株価、気温、売上高などが時系列データに該当します。
# 時系列データは順序が重要であり、時間を軸にして変動を追います。

# 時系列データの例（株価、気温、売上高） / Examples of Time Series Data
dates = pd.date_range(start='2022-01-01', periods=100, freq='D')
stock_prices = np.random.normal(100, 10, 100)  # ランダムな株価データ / Random stock prices
temperature = np.random.normal(30, 5, 100)  # ランダムな気温データ / Random temperature data
sales = np.random.normal(200, 50, 100)  # ランダムな売上データ / Random sales data

# ------------------------------
# 2. 時系列データの基本的な特性
# ------------------------------
# トレンド（Trend）：長期的な増減傾向 / Trend: Long-term increase or decrease
# 季節性（Seasonality）：周期的なパターン / Seasonality: Periodic patterns
# ノイズ（Noise）：ランダムな変動 / Noise: Random fluctuations
# サイクル（Cycles）：長期的な波動 / Cycles: Long-term fluctuations

# トレンドを加えたデータ例 / Example of data with a trend
time = np.arange(100)
trend = time * 0.5 + np.random.normal(0, 1, 100)  # トレンド（上昇） / Trend (rising)
seasonality = np.sin(2 * np.pi * time / 10) * 10  # 季節性 / Seasonality
noise = np.random.normal(0, 2, 100)  # ノイズ / Noise

# 時系列データの合成 / Combine these to create time series data
synthetic_data = trend + seasonality + noise

# ------------------------------
# 3. 時系列データの前処理
# ------------------------------
# データのスムージング / Data Smoothing (移動平均)
def moving_average(data, window_size=5):
    return np.convolve(data, np.ones(window_size)/window_size, mode='valid')

# 季節調整（Seasonal Decomposition）
# 季節調整のためにstatsmodelsを使用 / Seasonal decomposition using statsmodels
from statsmodels.tsa.seasonal import seasonal_decompose
series = pd.Series(synthetic_data)
result = seasonal_decompose(series, model='additive', period=10)
result.plot()
plt.show()

# 欠損値処理と外れ値処理 / Missing Value and Outlier Handling
# 欠損値処理 / Handling missing values
series_with_na = series.copy()
series_with_na[10:20] = np.nan  # 一部の値を欠損させる / Introduce NaN values
series_with_na.fillna(method='ffill', inplace=True)  # 前方補完 / Forward fill

# 外れ値処理 / Handling outliers
outliers = np.where(np.abs(synthetic_data - np.mean(synthetic_data)) > 3 * np.std(synthetic_data))[0]
synthetic_data[outliers] = np.mean(synthetic_data)  # 外れ値を平均値で置換 / Replace outliers with mean

# ------------------------------
# 4. 可視化
# ------------------------------
# 時系列データのプロット / Plotting Time Series Data
plt.figure(figsize=(10, 6))
plt.plot(time, synthetic_data, label='Synthetic Data with Trend, Seasonality, Noise')
plt.title('Time Series Data (Trend + Seasonality + Noise)')
plt.xlabel('Time')
plt.ylabel('Value')
plt.legend()
plt.grid(True)
plt.show()

# 自己相関・偏自己相関関数の可視化 / ACF and PACF Visualization
plt.figure(figsize=(12, 6))
plt.subplot(1, 2, 1)
plot_acf(synthetic_data, lags=40, ax=plt.gca())
plt.title('ACF (AutoCorrelation Function)')
plt.subplot(1, 2, 2)
plot_pacf(synthetic_data, lags=40, ax=plt.gca())
plt.title('PACF (Partial AutoCorrelation Function)')
plt.tight_layout()
plt.show()

# ヒストグラムや箱ひげ図での分析 / Histogram and Boxplot for Analysis
plt.figure(figsize=(10, 6))
plt.subplot(1, 2, 1)
plt.hist(synthetic_data, bins=20, color='skyblue', edgecolor='black')
plt.title('Histogram of Synthetic Data')
plt.subplot(1, 2, 2)
plt.boxplot(synthetic_data)
plt.title('Boxplot of Synthetic Data')
plt.tight_layout()
plt.show()

# ------------------------------
# 5. FFT（高速フーリエ変換）を使ったフィルタリング
# ------------------------------
# FFTによる周波数解析 / Frequency Analysis using FFT
def plot_fft(data):
    n = len(data)
    fft_vals = fft(data)
    fft_freq = np.fft.fftfreq(n)
    plt.plot(fft_freq, np.abs(fft_vals))
    plt.title('FFT of the Signal')
    plt.xlabel('Frequency')
    plt.ylabel('Magnitude')
    plt.grid(True)
    plt.show()

# ローパスフィルタ / Low-pass Filter
def low_pass_filter(data, cutoff_freq=0.1):
    b, a = butter(4, cutoff_freq, btype='low')
    return filtfilt(b, a, data)

# ハイパスフィルタ / High-pass Filter
def high_pass_filter(data, cutoff_freq=0.1):
    b, a = butter(4, cutoff_freq, btype='high')
    return filtfilt(b, a, data)

# バンドパスフィルタ / Band-pass Filter
def band_pass_filter(data, low_cutoff=0.1, high_cutoff=0.3):
    b, a = butter(4, [low_cutoff, high_cutoff], btype='bandpass')
    return filtfilt(b, a, data)

# FFTプロット / Plot FFT
plot_fft(synthetic_data)

# フィルタリングの適用 / Applying Filters
low_passed_data = low_pass_filter(synthetic_data)
high_passed_data = high_pass_filter(synthetic_data)
band_passed_data = band_pass_filter(synthetic_data)

# フィルタ後のデータプロット / Plot Filtered Data
plt.figure(figsize=(12, 6))
plt.plot(time, synthetic_data, label='Original Data')
plt.plot(time, low_passed_data, label='Low-pass Filtered', linestyle='--')
plt.plot(time, high_passed_data, label='High-pass Filtered', linestyle='--')
plt.plot(time, band_passed_data, label='Band-pass Filtered', linestyle='--')
plt.title('Filtered Time Series Data')
plt.xlabel('Time')
plt.ylabel('Value')
plt.legend()
plt.grid(True)
plt.show()

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris, make_classification
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LinearRegression, Ridge, Lasso, LogisticRegression
from sklearn.neighbors import KNeighborsClassifier
from sklearn.tree import DecisionTreeClassifier
from sklearn.ensemble import RandomForestClassifier
from sklearn.svm import SVC
from sklearn.naive_bayes import GaussianNB
from sklearn.preprocessing import StandardScaler
from sklearn.metrics import accuracy_score, mean_squared_error, f1_score, roc_curve, auc
from sklearn.decomposition import PCA
from sklearn.cluster import KMeans
from sklearn.svm import OneClassSVM
from sklearn.ensemble import IsolationForest

# ------------------------------
# 1. データセットの準備 / Dataset Preparation
# ------------------------------
# Irisデータセット / Iris dataset
iris = load_iris()
X = iris.data
y = iris.target

# データセットを訓練データとテストデータに分割 / Split dataset into training and test sets
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# 特徴量のスケーリング / Feature scaling
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

# ------------------------------
# 2. 教師あり学習 - 回帰 / Supervised Learning - Regression
# ------------------------------
# 線形回帰 / Linear Regression
linear_regressor = LinearRegression()
linear_regressor.fit(X_train_scaled, y_train)
y_pred_lr = linear_regressor.predict(X_test_scaled)
mse_lr = mean_squared_error(y_test, y_pred_lr)  # 回帰の評価指標 / Regression evaluation metric

# リッジ回帰 / Ridge Regression
ridge_regressor = Ridge(alpha=1.0)
ridge_regressor.fit(X_train_scaled, y_train)
y_pred_ridge = ridge_regressor.predict(X_test_scaled)
mse_ridge = mean_squared_error(y_test, y_pred_ridge)

# ラッソ回帰 / Lasso Regression
lasso_regressor = Lasso(alpha=0.1)
lasso_regressor.fit(X_train_scaled, y_train)
y_pred_lasso = lasso_regressor.predict(X_test_scaled)
mse_lasso = mean_squared_error(y_test, y_pred_lasso)

# ------------------------------
# 2. 教師あり学習 - 分類 / Supervised Learning - Classification
# ------------------------------
# ロジスティック回帰 / Logistic Regression
logistic_regressor = LogisticRegression(max_iter=200)
logistic_regressor.fit(X_train_scaled, y_train)
y_pred_log = logistic_regressor.predict(X_test_scaled)

# k-近傍法 / K-Nearest Neighbors
knn = KNeighborsClassifier(n_neighbors=3)
knn.fit(X_train_scaled, y_train)
y_pred_knn = knn.predict(X_test_scaled)

# 決定木 / Decision Tree
decision_tree = DecisionTreeClassifier(random_state=42)
decision_tree.fit(X_train_scaled, y_train)
y_pred_dt = decision_tree.predict(X_test_scaled)

# ランダムフォレスト / Random Forest
random_forest = RandomForestClassifier(n_estimators=100, random_state=42)
random_forest.fit(X_train_scaled, y_train)
y_pred_rf = random_forest.predict(X_test_scaled)

# サポートベクターマシン / Support Vector Machine (SVM)
svm = SVC(probability=True, random_state=42)
svm.fit(X_train_scaled, y_train)
y_pred_svm = svm.predict(X_test_scaled)

# ナイーブベイズ / Naive Bayes
naive_bayes = GaussianNB()
naive_bayes.fit(X_train_scaled, y_train)
y_pred_nb = naive_bayes.predict(X_test_scaled)

# ------------------------------
# 3. モデル評価 / Model Evaluation
# ------------------------------
# 精度 / Accuracy for classification models
accuracy_log = accuracy_score(y_test, y_pred_log)
accuracy_knn = accuracy_score(y_test, y_pred_knn)
accuracy_dt = accuracy_score(y_test, y_pred_dt)
accuracy_rf = accuracy_score(y_test, y_pred_rf)
accuracy_svm = accuracy_score(y_test, y_pred_svm)
accuracy_nb = accuracy_score(y_test, y_pred_nb)

# F1スコア / F1 Score for classification models
f1_log = f1_score(y_test, y_pred_log, average='weighted')
f1_knn = f1_score(y_test, y_pred_knn, average='weighted')
f1_dt = f1_score(y_test, y_pred_dt, average='weighted')
f1_rf = f1_score(y_test, y_pred_rf, average='weighted')
f1_svm = f1_score(y_test, y_pred_svm, average='weighted')
f1_nb = f1_score(y_test, y_pred_nb, average='weighted')

# 結果表示 / Display Results
print("教師あり学習 - 回帰評価 / Supervised Learning - Regression Metrics")
print(f"線形回帰 MSE / Linear Regression MSE: {mse_lr:.4f}")
print(f"リッジ回帰 MSE / Ridge Regression MSE: {mse_ridge:.4f}")
print(f"ラッソ回帰 MSE / Lasso Regression MSE: {mse_lasso:.4f}")

print("\n教師あり学習 - 分類評価 / Supervised Learning - Classification Metrics")
print("精度 / Accuracy:")
print(f"Logistic Regression: {accuracy_log:.4f}")
print(f"K-Nearest Neighbors: {accuracy_knn:.4f}")
print(f"Decision Tree: {accuracy_dt:.4f}")
print(f"Random Forest: {accuracy_rf:.4f}")
print(f"SVM: {accuracy_svm:.4f}")
print(f"Naive Bayes: {accuracy_nb:.4f}")

print("\nF1スコア / F1 Score:")
print(f"Logistic Regression: {f1_log:.4f}")
print(f"K-Nearest Neighbors: {f1_knn:.4f}")
print(f"Decision Tree: {f1_dt:.4f}")
print(f"Random Forest: {f1_rf:.4f}")
print(f"SVM: {f1_svm:.4f}")
print(f"Naive Bayes: {f1_nb:.4f}")

# ------------------------------
# 4. 教師なし学習 - クラスタリング / Unsupervised Learning - Clustering
# ------------------------------
# k-meansクラスタリング / K-Means Clustering
kmeans = KMeans(n_clusters=3, random_state=42)
kmeans.fit(X)
y_kmeans = kmeans.predict(X)

# PCAによる次元削減 / PCA for Dimensionality Reduction
pca = PCA(n_components=2)
X_pca = pca.fit_transform(X)

# ------------------------------
# 5. 異常検知 / Anomaly Detection
# ------------------------------
# One-Class SVM
oc_svm = OneClassSVM(kernel='rbf', gamma='scale', nu=0.1)
oc_svm.fit(X_train_scaled)
y_pred_oc_svm = oc_svm.predict(X_test_scaled)

# 孤立森林 / Isolation Forest
iso_forest = IsolationForest(contamination=0.1, random_state=42)
iso_forest.fit(X_train_scaled)
y_pred_iso_forest = iso_forest.predict(X_test_scaled)

# 結果表示 / Display clustering and anomaly detection results
plt.figure(figsize=(10, 6))
plt.scatter(X_pca[:, 0], X_pca[:, 1], c=y_kmeans, cmap='viridis', label='k-means clustering')
plt.title("PCA for Dimensionality Reduction and Clustering")
plt.xlabel("PC1")
plt.ylabel("PC2")
plt.legend()
plt.show()

print("\n異常検知 / Anomaly Detection Results:")
print(f"One-Class SVM predictions: {y_pred_oc_svm}")
print(f"Isolation Forest predictions: {y_pred_iso_forest}")

import numpy as np
import matplotlib.pyplot as plt
from tensorflow.keras.datasets import mnist
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense, Flatten
from tensorflow.keras.utils import to_categorical
from sklearn.model_selection import train_test_split

# ------------------------------
# 1. MNISTデータセットの読み込み
# ------------------------------
# データを訓練データとテストデータに分けて読み込む / Load MNIST dataset and split into training and test sets
(X_train, y_train), (X_test, y_test) = mnist.load_data()

# データのスケーリング / Scaling the data to [0, 1] range
X_train = X_train / 255.0
X_test = X_test / 255.0

# ラベルのOne-Hotエンコーディング / One-hot encoding the labels
y_train = to_categorical(y_train, 10)  # 10クラスの分類問題 / 10 classes for classification
y_test = to_categorical(y_test, 10)

# ------------------------------
# 2. ニューラルネットワークの構築
# ------------------------------
model = Sequential()

# 入力層 / Input layer: 28x28の画像をフラット化 / Flatten 28x28 images to 1D vector of 784 features
model.add(Flatten(input_shape=(28, 28)))

# 隠れ層 / Hidden layer: 128ユニットの全結合層 / Fully connected layer with 128 units
model.add(Dense(128, activation='relu'))

# 出力層 / Output layer: 10クラスに対するソフトマックス関数 / Softmax activation for 10 classes
model.add(Dense(10, activation='softmax'))

# ------------------------------
# 3. モデルのコンパイル
# ------------------------------
# 損失関数としてクロスエントロピー、最適化アルゴリズムとしてAdamを使用
model.compile(optimizer='adam', loss='categorical_crossentropy', metrics=['accuracy'])

# ------------------------------
# 4. モデルの訓練
# ------------------------------
# 訓練データで学習、検証用データとして20%を使用 / Training with 20% of data used for validation
history = model.fit(X_train, y_train, epochs=5, batch_size=32, validation_split=0.2)

# ------------------------------
# 5. モデルの評価
# ------------------------------
# テストデータでモデルを評価
test_loss, test_acc = model.evaluate(X_test, y_test)

print(f"Test Loss: {test_loss:.4f}")
print(f"Test Accuracy: {test_acc:.4f}")

# ------------------------------
# 6. 予測の表示 / Visualize Predictions
# ------------------------------
# テストデータの中からランダムに10枚の画像を表示し、予測結果を表示
predictions = model.predict(X_test)

fig, axes = plt.subplots(2, 5, figsize=(10, 5))
axes = axes.ravel()

for i in np.arange(10):
    ax = axes[i]
    ax.imshow(X_test[i], cmap='gray')
    ax.set_title(f"Pred: {np.argmax(predictions[i])}")
    ax.axis('off')

plt.subplots_adjust(wspace=0.5)
plt.show()

import numpy as np
import matplotlib.pyplot as plt
from sklearn.model_selection import train_test_split
from sklearn.neural_network import MLPClassifier
from sklearn.metrics import accuracy_score, classification_report, confusion_matrix

# ------------------------------
# 1. 論理回路データセット / Logical Gate Dataset
# ------------------------------
# ANDゲート / AND gate
X_and = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])  # 入力 / Input
y_and = np.array([0, 0, 0, 1])  # 出力 / Output

# ORゲート / OR gate
X_or = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])  # 入力 / Input
y_or = np.array([0, 1, 1, 1])  # 出力 / Output

# XORゲート / XOR gate
X_xor = np.array([[0, 0], [0, 1], [1, 0], [1, 1]])  # 入力 / Input
y_xor = np.array([0, 1, 1, 0])  # 出力 / Output

# ------------------------------
# 2. データの分割 / Data Splitting
# ------------------------------
# ANDゲート、ORゲート、XORゲートの訓練データとテストデータに分割
X_and_train, X_and_test, y_and_train, y_and_test = train_test_split(X_and, y_and, test_size=0.2, random_state=42)
X_or_train, X_or_test, y_or_train, y_or_test = train_test_split(X_or, y_or, test_size=0.2, random_state=42)
X_xor_train, X_xor_test, y_xor_train, y_xor_test = train_test_split(X_xor, y_xor, test_size=0.2, random_state=42)

# ------------------------------
# 3. ニューラルネットワークモデル / Neural Network Model
# ------------------------------
# MLPClassifierを使用してニューラルネットワークを訓練 / Train neural network using MLPClassifier
model_and = MLPClassifier(hidden_layer_sizes=(4,), max_iter=1000, random_state=42)
model_and.fit(X_and_train, y_and_train)

model_or = MLPClassifier(hidden_layer_sizes=(4,), max_iter=1000, random_state=42)
model_or.fit(X_or_train, y_or_train)

model_xor = MLPClassifier(hidden_layer_sizes=(4,), max_iter=1000, random_state=42)
model_xor.fit(X_xor_train, y_xor_train)

# ------------------------------
# 4. モデル評価 / Model Evaluation
# ------------------------------
# ANDゲートの予測 / Predict for AND gate
y_and_pred = model_and.predict(X_and_test)
print("ANDゲートの評価 / AND Gate Evaluation:")
print(f"Accuracy: {accuracy_score(y_and_test, y_and_pred):.4f}")
print(classification_report(y_and_test, y_and_pred))
print(confusion_matrix(y_and_test, y_and_pred))

# ORゲートの予測 / Predict for OR gate
y_or_pred = model_or.predict(X_or_test)
print("\nORゲートの評価 / OR Gate Evaluation:")
print(f"Accuracy: {accuracy_score(y_or_test, y_or_pred):.4f}")
print(classification_report(y_or_test, y_or_pred))
print(confusion_matrix(y_or_test, y_or_pred))

# XORゲートの予測 / Predict for XOR gate
y_xor_pred = model_xor.predict(X_xor_test)
print("\nXORゲートの評価 / XOR Gate Evaluation:")
print(f"Accuracy: {accuracy_score(y_xor_test, y_xor_pred):.4f}")
print(classification_report(y_xor_test, y_xor_pred))
print(confusion_matrix(y_xor_test, y_xor_pred))

# ------------------------------
# 5. 結果の可視化 / Visualization of Results
# ------------------------------
# ANDゲートの予測結果 / Visualizing results for AND gate
plt.figure(figsize=(6, 4))
plt.plot(y_and_test, label="True Values", marker='o')
plt.plot(y_and_pred, label="Predicted Values", marker='x')
plt.title('AND Gate: True vs Predicted')
plt.xlabel('Test Data Points')
plt.ylabel('Class')
plt.legend()
plt.show()

# ORゲートの予測結果 / Visualizing results for OR gate
plt.figure(figsize=(6, 4))
plt.plot(y_or_test, label="True Values", marker='o')
plt.plot(y_or_pred, label="Predicted Values", marker='x')
plt.title('OR Gate: True vs Predicted')
plt.xlabel('Test Data Points')
plt.ylabel('Class')
plt.legend()
plt.show()

# XORゲートの予測結果 / Visualizing results for XOR gate
plt.figure(figsize=(6, 4))
plt.plot(y_xor_test, label="True Values", marker='o')
plt.plot(y_xor_pred, label="Predicted Values", marker='x')
plt.title('XOR Gate: True vs Predicted')
plt.xlabel('Test Data Points')
plt.ylabel('Class')
plt.legend()
plt.show()

import numpy as np
import matplotlib.pyplot as plt

# 目的関数（損失関数）/ Objective (loss) function
def loss_function(x):
    return x**2 + 2*x + 1  # f(x) = (x+1)^2

# 勾配（導関数）/ Gradient of the loss function
def gradient(x):
    return 2*x + 2

# 各種最適化アルゴリズムの実装 / Implementations of gradient-based optimizers

# バッチ勾配降下法 / Batch Gradient Descent
def batch_gradient_descent(lr=0.1, epochs=30):
    x = 5.0
    history = [x]
    for _ in range(epochs):
        grad = gradient(x)
        x -= lr * grad
        history.append(x)
    return history

# モーメンタム法 / Momentum
def momentum_method(lr=0.1, epochs=30, beta=0.9):
    x = 5.0
    v = 0
    history = [x]
    for _ in range(epochs):
        grad = gradient(x)
        v = beta * v + (1 - beta) * grad
        x -= lr * v
        history.append(x)
    return history

# AdaGrad
def adagrad_method(lr=0.5, epochs=30, epsilon=1e-8):
    x = 5.0
    G = 0
    history = [x]
    for _ in range(epochs):
        grad = gradient(x)
        G += grad**2
        x -= (lr / (np.sqrt(G) + epsilon)) * grad
        history.append(x)
    return history

# RMSprop
def rmsprop_method(lr=0.1, epochs=30, beta=0.9, epsilon=1e-8):
    x = 5.0
    v = 0
    history = [x]
    for _ in range(epochs):
        grad = gradient(x)
        v = beta * v + (1 - beta) * grad**2
        x -= (lr / (np.sqrt(v) + epsilon)) * grad
        history.append(x)
    return history

# Adam
def adam_method(lr=0.1, epochs=30, beta1=0.9, beta2=0.999, epsilon=1e-8):
    x = 5.0
    m = 0
    v = 0
    history = [x]
    for t in range(1, epochs + 1):
        grad = gradient(x)
        m = beta1 * m + (1 - beta1) * grad
        v = beta2 * v + (1 - beta2) * grad**2
        m_hat = m / (1 - beta1**t)
        v_hat = v / (1 - beta2**t)
        x -= (lr / (np.sqrt(v_hat) + epsilon)) * m_hat
        history.append(x)
    return history

# Nadam
def nadam_method(lr=0.1, epochs=30, beta1=0.9, beta2=0.999, epsilon=1e-8):
    x = 5.0
    m = 0
    v = 0
    history = [x]
    for t in range(1, epochs + 1):
        grad = gradient(x)
        m = beta1 * m + (1 - beta1) * grad
        v = beta2 * v + (1 - beta2) * grad**2
        m_hat = m / (1 - beta1**t)
        v_hat = v / (1 - beta2**t)
        m_nesterov = beta1 * m_hat + (1 - beta1) * grad / (1 - beta1**t)
        x -= (lr / (np.sqrt(v_hat) + epsilon)) * m_nesterov
        history.append(x)
    return history

# 各アルゴリズムの実行
histories = {
    "Batch GD": batch_gradient_descent(),
    "Momentum": momentum_method(),
    "AdaGrad": adagrad_method(),
    "RMSprop": rmsprop_method(),
    "Adam": adam_method(),
    "Nadam": nadam_method()
}

# 結果のプロット
plt.figure(figsize=(10, 6))
for name, hist in histories.items():
    plt.plot(hist, label=name)

plt.title("Parameter Updates by Gradient-Based Optimization Algorithms")
plt.xlabel("Epoch")
plt.ylabel("x value")
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()

# --- Install required libraries ---
!pip install torch torchvision torchaudio --quiet

# --- Import libraries ---
import torch
import torch.nn as nn
import torch.optim as optim
from torchvision import datasets, transforms
from torch.utils.data import DataLoader
from torchtext.datasets import AG_NEWS
from torchtext.data.utils import get_tokenizer
from torchtext.vocab import build_vocab_from_iterator

# --- 1. CNN: MNIST image classification (Backprop + SGD) ---
# Define CNN architecture
class SimpleCNN(nn.Module):
    def __init__(self):
        super().__init__()
        self.conv = nn.Sequential(
            nn.Conv2d(1, 16, 3, 1),  # input, output, kernel, stride
            nn.ReLU(),
            nn.MaxPool2d(2)
        )
        self.fc = nn.Sequential(
            nn.Flatten(),
            nn.Linear(16*13*13, 128),
            nn.ReLU(),
            nn.Linear(128, 10)
        )

    def forward(self, x):
        x = self.conv(x)
        x = self.fc(x)
        return x

# Data loading for MNIST
transform = transforms.Compose([transforms.ToTensor()])
train_data = datasets.MNIST(root='.', train=True, download=True, transform=transform)
train_loader = DataLoader(train_data, batch_size=64, shuffle=True)

# Train CNN
cnn = SimpleCNN()
criterion = nn.CrossEntropyLoss()
optimizer = optim.SGD(cnn.parameters(), lr=0.01)

for epoch in range(1):
    for images, labels in train_loader:
        outputs = cnn(images)
        loss = criterion(outputs, labels)
        optimizer.zero_grad()
        loss.backward()  # ← Backpropagation
        optimizer.step()
    print(f"[CNN] Epoch {epoch+1}, Loss: {loss.item():.4f}")

# --- 2. RNN: AG_NEWS text classification ---
# RNN Architecture
class SimpleRNN(nn.Module):
    def __init__(self, vocab_size, embed_dim, hidden_size, output_size):
        super().__init__()
        self.embedding = nn.Embedding(vocab_size, embed_dim)
        self.rnn = nn.RNN(embed_dim, hidden_size, batch_first=True)
        self.fc = nn.Linear(hidden_size, output_size)

    def forward(self, x):
        embed = self.embedding(x)
        out, _ = self.rnn(embed)
        return self.fc(out[:, -1, :])  # Use last output

# Tokenizer and vocabulary
tokenizer = get_tokenizer("basic_english")

def yield_tokens(data_iter):
    for label, text in data_iter:
        yield tokenizer(text)

vocab = build_vocab_from_iterator(yield_tokens(AG_NEWS(split='train')), specials=["<unk>"])
vocab.set_default_index(vocab["<unk>"])

# Encode and collate
def collate_batch(batch):
    labels, texts = [], []
    for label, text in batch:
        labels.append(label - 1)
        texts.append(torch.tensor(vocab(tokenizer(text)), dtype=torch.int64))
    labels = torch.tensor(labels)
    texts = nn.utils.rnn.pad_sequence(texts, batch_first=True)
    return texts, labels

train_iter = AG_NEWS(split='train')
train_loader_rnn = DataLoader(list(train_iter)[:5000], batch_size=16, shuffle=True, collate_fn=collate_batch)

# Train RNN
rnn = SimpleRNN(vocab_size=len(vocab), embed_dim=64, hidden_size=32, output_size=4)
criterion = nn.CrossEntropyLoss()
optimizer = optim.Adam(rnn.parameters(), lr=0.001)

for epoch in range(1):
    for texts, labels in train_loader_rnn:
        outputs = rnn(texts)
        loss = criterion(outputs, labels)
        optimizer.zero_grad()
        loss.backward()  # ← Backpropagation
        optimizer.step()
    print(f"[RNN] Epoch {epoch+1}, Loss: {loss.item():.4f}")

# -*- coding: utf-8 -*-
"""
Unified Python Code for Linear Algebra Concepts
Print statements in English / コメントは日本語と英語併記
"""

import numpy as np
import matplotlib.pyplot as plt
from numpy.linalg import matrix_rank, eig, svd, qr
from sklearn.decomposition import PCA
from sklearn.linear_model import LinearRegression

# -------------------------------
# 2. Vector Space / ベクトル空間
# -------------------------------
print("=== 2.1 Vector Subspace and Independence ===")
V = np.array([[1, 2, 3], [2, 4, 6], [1, 0, 1]])
rank_V = matrix_rank(V)
print(f"Rank of V: {rank_V}")
if rank_V < V.shape[0]:
    print("The vectors are linearly dependent.")
else:
    print("The vectors are linearly independent.")

# -------------------------------
# 3. Determinant / 行列式
# -------------------------------
print("\n=== 3.1 Determinant ===")
A = np.array([[4, 2], [3, 1]])
det_A = np.linalg.det(A)
print(f"Determinant of A: {det_A:.2f}")

# -------------------------------
# 4. Eigenvalues and Eigenvectors / 固有値・固有ベクトル
# -------------------------------
print("\n=== 4.1 Eigenvalues and Eigenvectors ===")
B = np.array([[2, 0], [0, 3]])
vals, vecs = eig(B)
print("Eigenvalues:", vals)
print("Eigenvectors:\n", vecs)

# -------------------------------
# 5. Matrix Decomposition / 行列の分解
# -------------------------------
print("\n=== 5.1 QR and SVD Decomposition ===")
C = np.array([[1, 2], [3, 4]])
Q, R = qr(C)
U, S, VT = svd(C)
print("Q (from QR decomposition):\n", Q)
print("R (from QR decomposition):\n", R)
print("Singular values (from SVD):", S)

# -------------------------------
# 6.1 Least Squares Regression / 最小二乗法による回帰
# -------------------------------
print("\n=== 6.1 Linear Regression ===")
np.random.seed(0)
X = np.random.rand(100, 1) * 10
y = 2.5 * X.squeeze() + np.random.randn(100) * 2

model = LinearRegression()
model.fit(X, y)
print(f"Coefficient: {model.coef_[0]:.2f}, Intercept: {model.intercept_:.2f}")

# 回帰直線プロット / Plot regression line
plt.figure(figsize=(6, 4))
plt.scatter(X, y, alpha=0.6, label='Data')
plt.plot(X, model.predict(X), color='red', label='Regression Line')
plt.title("Linear Regression")
plt.xlabel("X")
plt.ylabel("y")
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()

# -------------------------------
# 6.2 Principal Component Analysis / 主成分分析
# -------------------------------
print("\n=== 6.2 Principal Component Analysis (PCA) ===")
X_pca = np.random.rand(100, 3)
pca = PCA(n_components=2)
X_transformed = pca.fit_transform(X_pca)
print("Explained Variance Ratio:", pca.explained_variance_ratio_)

# PCA結果の可視化 / PCA result visualization
plt.figure(figsize=(6, 4))
plt.scatter(X_transformed[:, 0], X_transformed[:, 1], alpha=0.6)
plt.title("PCA Projection")
plt.xlabel("PC1")
plt.ylabel("PC2")
plt.grid(True)
plt.tight_layout()
plt.show()

# 再実行用：ライブラリのインポートとすべてのコード統合

import numpy as np
import matplotlib.pyplot as plt
from scipy.stats import norm, beta
import seaborn as sns

# -------------------------------------------
# 1. ベイズ推定：コインの表確率の事後分布（Beta分布）
# -------------------------------------------
np.random.seed(0)
heads = 7
tails = 3
alpha_prior = 1
beta_prior = 1

# 事後分布は Beta(α + 表の回数, β + 裏の回数)
posterior = beta(alpha_prior + heads, beta_prior + tails)
x = np.linspace(0, 1, 100)
posterior_pdf = posterior.pdf(x)

plt.figure(figsize=(6, 4))
plt.plot(x, posterior_pdf, label='Posterior: Beta(8, 4)')
plt.title("Bayesian Inference: Posterior Distribution")
plt.xlabel("Probability of Heads")
plt.ylabel("Density")
plt.grid(True)
plt.legend()
plt.tight_layout()
plt.show()

# -------------------------------------------
# 2. MCMC: Metropolis-Hastingsで標準正規分布をサンプリング
# -------------------------------------------
def target_dist(x):
    return norm.pdf(x, loc=0, scale=1)

samples = []
x_curr = 0.0
for _ in range(5000):
    x_new = np.random.normal(x_curr, 1.0)
    accept_ratio = min(1, target_dist(x_new) / target_dist(x_curr))
    if np.random.rand() < accept_ratio:
        x_curr = x_new
    samples.append(x_curr)

plt.figure(figsize=(6, 4))
sns.histplot(samples, bins=50, kde=True, color='skyblue')
plt.title("Metropolis-Hastings Sampling (Normal Distribution)")
plt.xlabel("x")
plt.ylabel("Frequency")
plt.tight_layout()
plt.show()

# -------------------------------------------
# 3. 隠れマルコフモデル（HMM）: 生成・可視化（簡易版）
# -------------------------------------------
states = ['Rainy', 'Sunny']
observations = ['walk', 'shop', 'clean']
start_prob = {'Rainy': 0.6, 'Sunny': 0.4}
trans_prob = {
    'Rainy': {'Rainy': 0.7, 'Sunny': 0.3},
    'Sunny': {'Rainy': 0.4, 'Sunny': 0.6}
}
emission_prob = {
    'Rainy': {'walk': 0.1, 'shop': 0.4, 'clean': 0.5},
    'Sunny': {'walk': 0.6, 'shop': 0.3, 'clean': 0.1}
}

def generate_hmm_sequence(length=10):
    current_state = np.random.choice(states, p=[start_prob[s] for s in states])
    state_seq = [current_state]
    obs_seq = [np.random.choice(observations, p=[emission_prob[current_state][o] for o in observations])]

    for _ in range(length - 1):
        current_state = np.random.choice(states, p=[trans_prob[current_state][s] for s in states])
        state_seq.append(current_state)
        obs_seq.append(np.random.choice(observations, p=[emission_prob[current_state][o] for o in observations]))
    
    return state_seq, obs_seq

hidden_states, observations_seq = generate_hmm_sequence()

# 出力結果の表示
print("=== Hidden Markov Model (HMM) ===")
print("Hidden States:", hidden_states)
print("Observations:", observations_seq)

# -*- coding: utf-8 -*-
"""
統合された教師あり・教師なし学習アルゴリズムのデモ / Unified demos of supervised and unsupervised learning algorithms
"""

import numpy as np
import matplotlib.pyplot as plt
from sklearn.datasets import make_classification, make_regression, make_blobs
from sklearn.model_selection import train_test_split
from sklearn.preprocessing import StandardScaler
from sklearn.decomposition import PCA
from sklearn.manifold import TSNE
from sklearn.cluster import KMeans, DBSCAN, AgglomerativeClustering
from sklearn.mixture import GaussianMixture
from sklearn.linear_model import LogisticRegression, LinearRegression, Ridge, Lasso
from sklearn.svm import SVC, SVR
from sklearn.tree import DecisionTreeClassifier, DecisionTreeRegressor
from sklearn.ensemble import RandomForestClassifier, RandomForestRegressor, GradientBoostingRegressor
from sklearn.naive_bayes import GaussianNB
from sklearn.neighbors import KNeighborsClassifier
from sklearn.metrics import accuracy_score, mean_squared_error
from keras.models import Sequential
from keras.layers import Dense

# -------------------------------
# 1. 分類 / Classification
# -------------------------------
X_cls, y_cls = make_classification(n_samples=300, n_features=10, random_state=0)
X_train_cls, X_test_cls, y_train_cls, y_test_cls = train_test_split(X_cls, y_cls, test_size=0.2, random_state=42)

models_cls = {
    "Logistic Regression": LogisticRegression(),
    "SVM": SVC(),
    "K-NN": KNeighborsClassifier(),
    "Decision Tree": DecisionTreeClassifier(),
    "Random Forest": RandomForestClassifier(),
    "Naive Bayes": GaussianNB()
}

results_cls = {}
for name, model in models_cls.items():
    model.fit(X_train_cls, y_train_cls)
    preds = model.predict(X_test_cls)
    results_cls[name] = accuracy_score(y_test_cls, preds)

# -------------------------------
# 2. 回帰 / Regression
# -------------------------------
X_reg, y_reg = make_regression(n_samples=300, n_features=10, noise=10, random_state=0)
X_train_reg, X_test_reg, y_train_reg, y_test_reg = train_test_split(X_reg, y_reg, test_size=0.2, random_state=42)

models_reg = {
    "Linear Regression": LinearRegression(),
    "Ridge Regression": Ridge(),
    "Lasso Regression": Lasso(),
    "SVR": SVR(),
    "Decision Tree Regressor": DecisionTreeRegressor(),
    "Random Forest Regressor": RandomForestRegressor(),
    "Gradient Boosting": GradientBoostingRegressor()
}

results_reg = {}
for name, model in models_reg.items():
    model.fit(X_train_reg, y_train_reg)
    preds = model.predict(X_test_reg)
    results_reg[name] = mean_squared_error(y_test_reg, preds)

# -------------------------------
# 3. 教師なし学習 / Unsupervised Learning
# -------------------------------
X_unsup, _ = make_blobs(n_samples=300, centers=3, n_features=5, random_state=0)
X_unsup_scaled = StandardScaler().fit_transform(X_unsup)

# PCA for Visualization
X_pca = PCA(n_components=2).fit_transform(X_unsup_scaled)

# クラスタリング / Clustering
clustering_models = {
    "KMeans": KMeans(n_clusters=3),
    "DBSCAN": DBSCAN(eps=0.5),
    "Agglomerative": AgglomerativeClustering(n_clusters=3),
    "GMM": GaussianMixture(n_components=3)
}

plt.figure(figsize=(12, 8))
for i, (name, model) in enumerate(clustering_models.items()):
    if hasattr(model, 'fit_predict'):
        labels = model.fit_predict(X_unsup_scaled)
    else:
        model.fit(X_unsup_scaled)
        labels = model.predict(X_unsup_scaled)
    plt.subplot(2, 2, i+1)
    plt.scatter(X_pca[:, 0], X_pca[:, 1], c=labels, cmap='Set1')
    plt.title(f"{name} Clustering (PCA)")
plt.tight_layout()
plt.show()

# -------------------------------
# 結果表示 / Display results
# -------------------------------
print("=== Classification Accuracies / 分類精度 ===")
for name, score in results_cls.items():
    print(f"{name}: {score:.2f}")

print("\n=== Regression MSE / 回帰の平均二乗誤差 ===")
for name, score in results_reg.items():
    print(f"{name}: {score:.2f}")

import numpy as np
import pandas as pd
import matplotlib.pyplot as plt
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split, GridSearchCV, RandomizedSearchCV, cross_val_score
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import accuracy_score, f1_score, confusion_matrix, roc_curve, auc
from sklearn.model_selection import StratifiedKFold
from sklearn.preprocessing import StandardScaler
from scipy.stats import randint

# ------------------------------
# 1. データの準備 / Load Dataset
# ------------------------------
# Irisデータセットを使います / Using the Iris dataset
iris = load_iris()
X = iris.data
y = iris.target

# 訓練データとテストデータに分割 / Split into train and test data
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.3, random_state=42)

# 特徴量を標準化 / Standardize features
scaler = StandardScaler()
X_train_scaled = scaler.fit_transform(X_train)
X_test_scaled = scaler.transform(X_test)

# ------------------------------
# 2. ハイパーパラメータ調整 / Hyperparameter Tuning
# ------------------------------
# グリッドサーチ / Grid Search
param_grid = {
    'n_estimators': [50, 100, 200],
    'max_depth': [None, 10, 20],
    'min_samples_split': [2, 5],
    'min_samples_leaf': [1, 2, 4]
}

grid_search = GridSearchCV(RandomForestClassifier(random_state=42), param_grid, cv=5, n_jobs=-1)
grid_search.fit(X_train_scaled, y_train)
print("Best parameters from Grid Search:", grid_search.best_params_)

# ランダムサーチ / Random Search
param_dist = {
    'n_estimators': randint(50, 200),
    'max_depth': [None, 10, 20],
    'min_samples_split': randint(2, 10),
    'min_samples_leaf': randint(1, 5)
}

random_search = RandomizedSearchCV(RandomForestClassifier(random_state=42), param_dist, n_iter=100, cv=5, n_jobs=-1, random_state=42)
random_search.fit(X_train_scaled, y_train)
print("Best parameters from Random Search:", random_search.best_params_)

# ------------------------------
# 3. 交差検証 / Cross-Validation
# ------------------------------
# 交差検証での精度評価 / Cross-validation accuracy
cv = StratifiedKFold(n_splits=5, shuffle=True, random_state=42)
cross_val_scores = cross_val_score(RandomForestClassifier(n_estimators=100, max_depth=10, random_state=42), X_train_scaled, y_train, cv=cv, scoring='accuracy')
print(f"Cross-validation Accuracy: {cross_val_scores.mean():.4f} ± {cross_val_scores.std():.4f}")

# ------------------------------
# 4. モデル評価 / Model Evaluation
# ------------------------------
# ベストなモデルでテストデータに対する予測 / Prediction using the best model
best_model = random_search.best_estimator_
y_pred = best_model.predict(X_test_scaled)

# 精度 / Accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.4f}")

# F1スコア / F1 Score
f1 = f1_score(y_test, y_pred, average='weighted')
print(f"F1 Score: {f1:.4f}")

# 混同行列 / Confusion Matrix
cm = confusion_matrix(y_test, y_pred)
plt.figure(figsize=(6, 4))
plt.imshow(cm, interpolation='nearest', cmap=plt.cm.Blues)
plt.title("Confusion Matrix")
plt.colorbar()
plt.xticks(np.arange(3), iris.target_names)
plt.yticks(np.arange(3), iris.target_names)
plt.xlabel('Predicted label')
plt.ylabel('True label')
plt.tight_layout()
plt.show()

# ROC曲線とAUC / ROC Curve and AUC
# 2クラスの分類問題として評価 / Evaluating as a 2-class classification problem
y_test_binary = (y_test == 0)  # クラス0とそれ以外を分類 / Classify class 0 vs others
y_pred_prob = best_model.predict_proba(X_test_scaled)[:, 1]

fpr, tpr, _ = roc_curve(y_test_binary, y_pred_prob)
roc_auc = auc(fpr, tpr)

plt.figure(figsize=(6, 4))
plt.plot(fpr, tpr, color='blue', label=f'ROC curve (AUC = {roc_auc:.2f})')
plt.plot([0, 1], [0, 1], color='gray', linestyle='--')
plt.xlim([0.0, 1.0])
plt.ylim([0.0, 1.05])
plt.xlabel('False Positive Rate')
plt.ylabel('True Positive Rate')
plt.title('Receiver Operating Characteristic (ROC) Curve')
plt.legend(loc='lower right')
plt.show()


import numpy as np
from gensim.models import Word2Vec
from transformers import BertTokenizer, BertModel
import torch

# ------------------------------
# 1. Word2Vecの実装 / Word2Vec Implementation
# ------------------------------
# サンプルのテキストデータ / Sample text data
sentences = [
    ['i', 'love', 'machine', 'learning'],
    ['word2vec', 'is', 'a', 'technique', 'in', 'nlp'],
    ['natural', 'language', 'processing', 'is', 'fun']
]

# Word2Vecモデルの訓練 / Train Word2Vec model
word2vec_model = Word2Vec(sentences, vector_size=50, window=3, min_count=1, workers=4)

# 'machine' のベクトルを表示 / Display vector for 'machine'
word2vec_vector = word2vec_model.wv['machine']
print("Word2Vec ベクトル:", word2vec_vector)

# ------------------------------
# 2. GloVeの実装 / GloVe Implementation
# ------------------------------
# GloVeの事前訓練済みモデル（例: glove.6B.50d.txt）を読み込み / Load pre-trained GloVe model (e.g., glove.6B.50d.txt)
def load_glove_model(file_path):
    model = {}
    with open(file_path, 'r', encoding='utf-8') as f:
        for line in f:
            values = line.split()
            word = values[0]
            vector = np.asarray(values[1:], dtype='float32')
            model[word] = vector
    return model

# GloVeモデルをロード / Load GloVe model
# GloVeファイルのパスを指定してください / Specify the path to the GloVe file
glove_model = load_glove_model('glove.6B.50d.txt')  # glove.6B.50d.txt のパス

# 'king' のベクトルを表示 / Display vector for 'king'
glove_vector = glove_model.get('king', 'Not Found')
print("GloVe ベクトル:", glove_vector)

# ------------------------------
# 3. BERTの実装 / BERT Implementation
# ------------------------------
# BERTトークナイザーとモデルを読み込む / Load BERT tokenizer and model
tokenizer = BertTokenizer.from_pretrained('bert-base-uncased')
model = BertModel.from_pretrained('bert-base-uncased')

# 入力文 / Input text
input_text = "I love machine learning."

# トークン化 / Tokenize the input text
inputs = tokenizer(input_text, return_tensors="pt")

# BERTモデルで埋め込みを取得 / Get embeddings from BERT model
with torch.no_grad():
    outputs = model(**inputs)

# 最後の隠れ層の出力（[CLS]トークン） / Last hidden layer output (embedding of [CLS] token)
bert_embedding = outputs.last_hidden_state[0][0]
print("BERT 出力（[CLS]トークン）:", bert_embedding)

# ------------------------------
# 結果の表示 / Display results
# ------------------------------
print("\n--- 結果 ---")
print("Word2Vec 'machine' ベクトル:", word2vec_vector)
print("GloVe 'king' ベクトル:", glove_vector)
print("BERT [CLS] トークンの埋め込み:", bert_embedding)


# Program Name: linear_algebra_kernels_analysis.py
# Creation Date: 20250111
# Overview: Perform eigen decomposition, SVD, PCA, matrix differentiation, and kernel trick visualization
# Usage: To run the program, use the command `python linear_algebra_kernels_analysis.py` in the terminal

# --- ライブラリのインストール / Install required libraries ---
# ※Google Colab などで使用する際は先頭の ! を外してください
# !pip install numpy matplotlib scipy seaborn

import numpy as np
import matplotlib.pyplot as plt
from numpy.linalg import eig, svd
from sklearn.decomposition import PCA
from scipy import linalg
import seaborn as sns

# --- 数値の定義セクション / Define parameters in one place ---
A = np.array([[4, -2], [1, 1]])       # 固有値・固有ベクトルの対象行列 / Matrix for eigenvalues
M = np.array([[3, 1, 1], [-1, 3, 1]]) # SVD対象行列 / Matrix for SVD
X = np.random.randn(100, 2)          # PCA対象データ / Data matrix for PCA
x, y = np.meshgrid(np.linspace(-3, 3, 100), np.linspace(-3, 3, 100))  # カーネル可視化用

# --- 固有値・固有ベクトルとスペクトル分解 / Eigen decomposition ---
eigvals, eigvecs = eig(A)

# 日本語と英語の詳細コメント:
# 固有値と固有ベクトルは Ax = λx を満たすベクトルとスカラーの組です。
# Eigenvalues and eigenvectors satisfy Ax = λx.
print("Eigenvalues:", eigvals)
print("Eigenvectors:\n", eigvecs)

# --- 特異値分解（SVD） / Singular Value Decomposition ---
U, S, Vt = svd(M)

# U: 左特異ベクトル / Left singular vectors
# S: 特異値 / Singular values
# Vt: 右特異ベクトルの転置 / Transpose of right singular vectors
print("U:\n", U)
print("Singular values:", S)
print("V^T:\n", Vt)

# --- 主成分分析（PCA） / Principal Component Analysis ---
pca = PCA(n_components=2)
X_pca = pca.fit_transform(X)

# --- PCA 結果の可視化 / Visualize PCA result ---
plt.figure(figsize=(6, 5))
plt.scatter(X_pca[:, 0], X_pca[:, 1], alpha=0.7, c='teal')
plt.title("PCA Projection")
plt.xlabel("Principal Component 1")
plt.ylabel("Principal Component 2")
plt.grid(True)
plt.tight_layout()
plt.show()

# --- 行列の微分（例：2変数の2x2行列に関する勾配）/ Matrix differentiation example ---
# f(W) = Tr(W @ W.T) の勾配 → 2W
# Gradient of f(W) = Tr(W @ W.T) is ∂f/∂W = 2W
W = np.array([[2.0, -1.0], [0.0, 3.0]])
grad_f_W = 2 * W
print("Matrix W:\n", W)
print("Gradient of Tr(W @ W.T):\n", grad_f_W)

# --- カーネル法とRKHS（RBFカーネルの可視化） / Kernel Trick and RKHS visualization ---
# RBF (Gaussian) Kernel function
def rbf_kernel(x1, x2, gamma=0.5):
    diff = x1 - x2
    return np.exp(-gamma * np.dot(diff, diff))

# カーネル行列の構築 / Build kernel matrix on meshgrid
xy_stack = np.dstack([x, y])
center = np.array([0.0, 0.0])
Z = np.array([[rbf_kernel(np.array([xi, yi]), center) for xi, yi in row] for row in xy_stack])

# --- RBFカーネルのヒートマップ表示 / Plot RBF kernel heatmap ---
plt.figure(figsize=(6, 5))
sns.heatmap(Z, cmap='viridis')
plt.title("RBF Kernel Heatmap (Centered at Origin)")
plt.xlabel("X-axis")
plt.ylabel("Y-axis")
plt.tight_layout()
plt.show()

# プログラム名: probability_statistics_module.py
# 作成日: 20250111
# 概要: 確率論と統計に基づく基本的な分布、推定、ベイズの定理、マルコフ連鎖を実装・可視化
# 使用方法: `python probability_statistics_module.py` にて実行

# 必要なライブラリのインストール（Google Colab等では!を先頭に）
# !pip install numpy scipy matplotlib seaborn

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from scipy.stats import norm, expon, poisson, multivariate_normal
from numpy.linalg import inv
import random

# --- 数値設定 / Set parameters ---
mu = 0                      # 平均 / Mean for Normal
sigma = 1                   # 標準偏差 / Std Dev
lambda_exp = 1.0            # 指数分布のλ / Lambda for Exponential
lambda_poi = 3.0            # ポアソン分布のλ / Lambda for Poisson
samples = 1000              # サンプル数 / Number of samples

# --- 確率分布の描画 / Plot probability distributions ---
x = np.linspace(-5, 10, 1000)
plt.figure(figsize=(10, 6))
plt.plot(x, norm.pdf(x, mu, sigma), label='Normal')
plt.plot(x, expon.pdf(x, scale=1/lambda_exp), label='Exponential')
plt.plot(np.arange(15), poisson.pmf(np.arange(15), lambda_poi), label='Poisson', marker='o')
plt.title("Probability Distributions")
plt.xlabel("x")
plt.ylabel("Probability Density / Mass")
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()

# --- ベイズの定理 / Bayes Theorem ---
# P(A|B) = P(B|A) * P(A) / P(B)
P_A = 0.01    # 病気の人の割合 / Prior: having disease
P_B_given_A = 0.99  # 検査が陽性になる確率（病気のとき） / Likelihood
P_B_given_notA = 0.05  # 偽陽性率 / False Positive Rate
P_notA = 1 - P_A

P_B = P_B_given_A * P_A + P_B_given_notA * P_notA
P_A_given_B = (P_B_given_A * P_A) / P_B
print("P(A|B) =", P_A_given_B)

# --- 最尤推定とベイズ推定 / MLE vs Bayesian Estimation ---
data = np.random.normal(loc=5.0, scale=2.0, size=100)
mle_estimate = np.mean(data)  # 最尤推定 = サンプル平均 / MLE is sample mean

# ベイズ推定：事前分布 ~ N(μ0=0, σ0²=1), 観測値 ~ N(μ, σ²=4)
mu_0, sigma_0_sq = 0.0, 1.0
sigma_sq = 4.0
n = len(data)
sample_mean = np.mean(data)

# ベイズの事後平均 / Posterior Mean
posterior_mean = (sample_mean * n / sigma_sq + mu_0 / sigma_0_sq) / (n / sigma_sq + 1 / sigma_0_sq)
print("MLE Mean:", mle_estimate)
print("Bayesian Posterior Mean:", posterior_mean)

# --- 共分散と多変量正規分布 / Covariance and Multivariate Normal ---
mean_vec = np.array([0, 0])
cov_matrix = np.array([[1.0, 0.8], [0.8, 1.0]])  # 共分散 / Covariance
mv_data = np.random.multivariate_normal(mean_vec, cov_matrix, size=500)

plt.figure(figsize=(6, 5))
sns.scatterplot(x=mv_data[:, 0], y=mv_data[:, 1], alpha=0.5)
plt.title("Multivariate Normal Distribution")
plt.xlabel("X1")
plt.ylabel("X2")
plt.grid(True)
plt.tight_layout()
plt.show()

# --- マルコフ連鎖（簡易版）/ Simple Markov Chain ---
states = ['Rainy', 'Sunny']
transition_matrix = [[0.7, 0.3], [0.4, 0.6]]
state_dict = {'Rainy': 0, 'Sunny': 1}

current_state = 'Rainy'
chain = [current_state]

for _ in range(50):
    current_idx = state_dict[current_state]
    next_state = np.random.choice(states, p=transition_matrix[current_idx])
    chain.append(next_state)
    current_state = next_state

print("Markov Chain Simulation:\n", chain)

# Program Name: information_theory_analysis.py
# Creation Date: 20250111
# Overview: 計算・可視化を通じて情報理論の主要概念（エントロピー、クロスエントロピー、KL、情報ゲイン、ELBO）を理解するプログラム
# Usage: To run the program, use the command `python information_theory_analysis.py` in the terminal

# --- ライブラリのインストール（Google Colab等では ! を外してください） ---
# !pip install numpy matplotlib seaborn scikit-learn

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from sklearn.tree import DecisionTreeClassifier, plot_tree
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split
from sklearn.metrics import log_loss

# --- パラメータ定義 / Define parameters ---
p = np.array([0.1, 0.4, 0.5])  # 真の分布 / True distribution
q = np.array([0.2, 0.3, 0.5])  # 予測分布 / Predicted distribution

# --- エントロピー・クロスエントロピー・KL / Entropy, Cross-Entropy, KL Divergence ---
def entropy(p):
    return -np.sum(p * np.log2(p + 1e-10))  # エントロピー H(p)

def cross_entropy(p, q):
    return -np.sum(p * np.log2(q + 1e-10))  # クロスエントロピー H(p, q)

def kl_divergence(p, q):
    return np.sum(p * np.log2((p + 1e-10) / (q + 1e-10)))  # KL(p||q)

H_p = entropy(p)
H_pq = cross_entropy(p, q)
KL_pq = kl_divergence(p, q)

print(f"Entropy H(p): {H_p:.4f}")
print(f"Cross Entropy H(p, q): {H_pq:.4f}")
print(f"KL Divergence D_KL(p || q): {KL_pq:.4f}")

# --- 情報ゲインと決定木 / Information Gain with Decision Tree ---
iris = load_iris()
X_train, X_test, y_train, y_test = train_test_split(iris.data, iris.target, random_state=42)

clf = DecisionTreeClassifier(criterion="entropy", max_depth=3, random_state=42)
clf.fit(X_train, y_train)

plt.figure(figsize=(10, 6))
plot_tree(clf, feature_names=iris.feature_names, class_names=iris.target_names, filled=True)
plt.title("Decision Tree with Information Gain")
plt.tight_layout()
plt.show()

# --- 最小記述長（MDL）の考え方 / Minimum Description Length (conceptual simulation) ---
# モデル複雑度 + 残差情報量で最小となるモデルを選ぶ（AIC的考え）
model_complexities = np.array([1, 2, 3, 4, 5])
description_lengths = np.array([12, 9, 7, 8, 10])  # 仮の記述長合計

plt.figure(figsize=(6, 4))
plt.plot(model_complexities, description_lengths, marker='o')
plt.title("Minimum Description Length Principle")
plt.xlabel("Model Complexity")
plt.ylabel("Total Description Length")
plt.grid(True)
plt.tight_layout()
plt.show()

# --- ELBOと変分推論の直感的シミュレーション / ELBO Intuition ---
# 真の事後 p(z|x) はわからない → 近似分布 q(z) を使って下限を最大化
# ELBO = E_q[log p(x|z)] - KL(q(z) || p(z))

# シンプルなケースで比較：真のpとqはガウス分布
mu_p, sigma_p = 0, 1     # 真の分布 p(z)
mu_q, sigma_q = 1, 1.5   # 近似分布 q(z)

def kl_gaussian(mu_q, sigma_q, mu_p, sigma_p):
    return np.log(sigma_p / sigma_q) + (sigma_q**2 + (mu_q - mu_p)**2) / (2 * sigma_p**2) - 0.5

kl = kl_gaussian(mu_q, sigma_q, mu_p, sigma_p)

# ELBOの近似計算（事後期待は省略しKLの大小比較のみ）
print(f"Approximate KL Divergence (ELBO lower bound term): {kl:.4f}")

# Program Name: rkhs_geometry_variational.py
# Creation Date: 20250111
# Overview: 関数解析（RKHS）と微分幾何・変分法の数理を可視化・計算する統合モジュール
# Usage: To run the program, use the command `python rkhs_geometry_variational.py` in the terminal

# --- 必要なライブラリのインストール ---
# !pip install numpy matplotlib seaborn scipy

import numpy as np
import matplotlib.pyplot as plt
import seaborn as sns
from scipy.spatial.distance import cdist
from numpy.linalg import inv, norm

# --- 数値パラメータの集中定義 / Centralized parameters ---
gamma = 0.5  # RBFカーネルのγ / Gamma for RBF kernel
x = np.linspace(-3, 3, 100).reshape(-1, 1)
x_test = np.array([[0.0]])

# --- RBFカーネル定義とRKHS内積構造 / RBF kernel and inner product in RKHS ---
def rbf_kernel(x1, x2, gamma):
    return np.exp(-gamma * cdist(x1, x2, 'sqeuclidean'))

K = rbf_kernel(x, x, gamma)         # カーネル行列 / Kernel Gram matrix
k_x = rbf_kernel(x, x_test, gamma)  # 評価点へのカーネル / Kernel vector to test point
alpha = np.random.randn(len(x), 1)  # RKHS上の関数係数 / Coefficients for RKHS function

# 再生性：f(x) = ⟨f, K(x, ·)⟩_H を確認 / Reproducing property
f_hat = float(k_x.T @ alpha)

# --- RKHS関数のプロット / Plot RKHS function approximation ---
f_values = K @ alpha  # 再生核で構成される関数値 / Values of function f(x) in RKHS

plt.figure(figsize=(7, 4))
plt.plot(x, f_values, label="f(x) in RKHS")
plt.axvline(x=0.0, color='r', linestyle='--', label='Test point x=0')
plt.scatter(0, f_hat, color='red', label='f(0) = <f, K(0,·)>')
plt.title("Function in RKHS with RBF Kernel")
plt.xlabel("x")
plt.ylabel("f(x)")
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()

# --- Fisher情報行列と情報幾何 / Fisher Information Matrix ---
# 2項分布の例：p(x;θ) = θ^x (1-θ)^(1-x)
theta_vals = np.linspace(0.01, 0.99, 100)
fisher_info = 1 / (theta_vals * (1 - theta_vals))  # Fisher情報量

plt.figure(figsize=(6, 4))
plt.plot(theta_vals, fisher_info)
plt.title("Fisher Information (Bernoulli Distribution)")
plt.xlabel("θ")
plt.ylabel("Fisher Information I(θ)")
plt.grid(True)
plt.tight_layout()
plt.show()

# --- 変分法：エネルギー汎関数の最小化 / Calculus of Variations ---
# エネルギー汎関数 J[y] = ∫ (y')² dx を y(x=0)=0, y(x=1)=1 の下で最小化
# 解は直線 y(x) = x

x_var = np.linspace(0, 1, 100)
y_candidate = x_var + 0.1 * np.sin(5 * np.pi * x_var)  # 擾乱を加えた関数
energy = np.trapz((np.gradient(y_candidate, x_var))**2, x_var)

plt.figure(figsize=(6, 4))
plt.plot(x_var, y_candidate, label=f"Perturbed y(x), Energy = {energy:.3f}")
plt.plot(x_var, x_var, '--', label="True Minimizer y(x)=x")
plt.title("Energy Functional Minimization")
plt.xlabel("x")
plt.ylabel("y(x)")
plt.legend()
plt.grid(True)
plt.tight_layout()
plt.show()