The Transformer model is a groundbreaking technology in natural language processing, with the self-attention mechanism at its core. This mechanism starts by representing each word as a vector and applying linear transformations to generate three vectors: query, key, and value. The model calculates the dot product of the query and key vectors, scales the result by the square root of the vector dimension, and normalizes it using the softmax function to compute attention weights. These weights are then multiplied by the value vectors to quantify the relationships between all words in a sentence. By processing multiple attention heads in parallel through multi-head attention, the model captures diverse contextual information. All these operations are based on linear algebra techniques such as matrix multiplication and transposition, making the Transformer highly efficient for large-scale learning. Scaling up this architecture with billions of parameters trained on massive text data leads to Large Language Models (LLMs). LLMs utilize attention-based weighted linear combinations to process information and perform various tasks, including question answering, translation, summarization, and even generating programming code, as demonstrated by ChatGPT, which is now widely available to users.
- 未来表現（will / be going to）
文法機能 数学・機械学習文脈例文
未来の予定 We are going to solve the quadratic equation in tomorrow’s class.
未来の意志 I will build a logistic regression model to classify the data.
未来の予測 The model will improve after more training epochs.
確実な予定 Our team is going to compare linear and polynomial regression next week.
予測と根拠 The loss curve is going to flatten after epoch 50.
瞬間の決定 I will apply the chain rule now to differentiate this function.
計画された動作 We are going to prepare a dataset with 10,000 samples.
将来の改善案 We will use a different optimizer next time.
研究発表予定 I am going to present our gradient boosting results.
数学探究計画 I will investigate the behavior of the exponential function.
⸻
- 助動詞 can / must / should
文法機能 数学・機械学習文脈例文
可能 You can calculate the slope using the derivative.
義務 You must normalize the data before applying PCA.
助言 You should check the learning rate before training.
能力 This model can detect non-linear relationships.
絶対必要 You must define the loss function correctly.
推奨 You should visualize the decision boundary.
条件付き You can improve the accuracy if you add regularization.
制約 The batch size must be smaller than the dataset size.
技術選択 You should try both gradient descent and stochastic gradient descent.
設計基準 The function must satisfy the Lipschitz condition.
⸻
- 不定詞（to + 動詞）
文法機能 数学・機械学習文脈例文
目的 I implemented gradient descent to minimize the loss function.
感情理由 I am excited to apply eigenvalue decomposition in PCA.
説明 The goal is to improve generalization by adding dropout.
結果 We used cross-validation to select the best hyperparameters.
方法 You can use the chain rule to differentiate complex functions.
手順 The next step is to normalize the feature vectors.
目標設定 We decided to increase the number of training samples.
研究目的 Our project aims to reduce overfitting in deep neural networks.
実験設計 We conducted the experiment to measure classification performance.
課題解決 The model was modified to handle missing values.
⸻
- 動名詞（動詞 + ing）
文法機能 数学・機械学習文脈例文
主語 Regularizing the model prevents overfitting.
目的語 We recommend using early stopping during training.
手段 Applying normalization improves convergence.
習慣 Visualizing the data is part of our workflow.
継続動作 Adjusting the learning rate helps achieve better accuracy.
方法 You can improve the model by reducing the loss.
説明 Increasing the number of layers can make the model more powerful.
目的 We started exploring new activation functions.
研究活動 Comparing different optimizers is our current task.
工夫 Tuning hyperparameters takes time and experimentation.
⸻
- 受動態（be + 過去分詞）
文法機能 数学・機械学習文脈例文
受動の事実 The equation was derived using Taylor expansion.
手順説明 The data was split into training and test sets.
処理説明 The image was resized to 256 by 256 pixels.
設計結果 The model was trained on 1 million examples.
発見報告 A new loss function was proposed in the latest paper.
機能強調 The dataset was labeled by human annotators.
検証内容 The algorithm was validated on multiple benchmarks.
実装報告 The function was implemented using NumPy.
計算実績 The eigenvalues were calculated for the covariance matrix.
実験結果 The performance was improved by using batch normalization.
⸻
- 現在完了（have/has + 過去分詞）
文法機能 数学・機械学習文脈例文
経験 I have implemented various optimization algorithms.
継続 We have been studying eigenvalue problems for two weeks.
完了 The model has achieved 95% accuracy.
状況報告 We have trained the model 10 times so far.
成果報告 I have published a paper on loss function analysis.
状況確認 Have you checked the learning curve yet?
活動経過 The team has improved the architecture based on recent findings.
実装報告 We have tested the gradient boosting implementation.
データ取得 We have collected 10,000 handwritten digits for training.
実験実績 We have conducted 5 experiments to compare optimizers.
⸻
- 関係代名詞（who, which, that）
文法機能 数学・機械学習文脈例文
人を説明 The engineer who developed this algorithm works at Google.
物を説明 The function which minimizes the loss is called the cost function.
汎用説明 The method that uses batch normalization improves stability.
複数対象 The students who studied eigenvalues passed the test.
定義追加 The concept that describes non-linearity is called activation.
モデル説明 The model which is based on decision trees is called Random Forest.
結果説明 The result that surprised us was the low error rate.
状況描写 The framework that supports GPU acceleration is TensorFlow.
数学定義 A matrix which has equal rows and columns is called a square matrix.
関数性質 The function that increases monotonically is called increasing function.
⸻
- 仮定法過去・仮定法過去完了
文法機能 数学・機械学習文脈例文
仮想条件 If the dataset were larger, the model would perform better.
反事実 If we had used more features, the accuracy would have improved.
改善案提示 If the learning rate were smaller, training would be more stable.
結果反省 If we had validated properly, the model would not have overfitted.
追加条件 If the optimizer were Adam, convergence would be faster.
理想案 If we had had more computational resources, we could have tried larger models.
研究仮説 If the data were perfectly balanced, the classifier would perform ideally.
後悔表現 If I had realized the error earlier, I would have fixed the code.
検証不足 If we had tested more cases, we might have avoided the bug.
実験改善 If we had adjusted the regularization term, the loss could have been lower.
⸻
- 分詞構文（現在分詞・過去分詞）
文法機能 数学・機械学習文脈例文
方法説明 Using gradient descent, we optimized the cost function.
付帯状況 Considering the variance, we decided to apply standardization.
時間関係 Starting from zero, the algorithm gradually improved the weights.
結果説明 Obtaining high accuracy, we submitted our model to the competition.
状況設定 Given the large dataset, we used a distributed computing platform.
前提提示 Assuming the data is normalized, we applied PCA.
条件設定 Provided that the loss decreases, we continue training.
成果説明 Achieving convergence, we stopped the training early.
手法紹介 Combining decision trees, we created a Random Forest model.
状況展開 Faced with low accuracy, we decided to increase the dataset size.
⸻
- 強調構文（It is ~ that …）
文法機能 数学・機械学習文脈例文
特定強調 It is normalization that makes gradient descent more stable.
人物強調 It is the data scientist who proposed this model.
時間強調 It was last week that we discovered the implementation bug.
方法強調 It is feature scaling that improves convergence speed.
結果強調 It was the regularization term that reduced overfitting.
状況強調 It is when the data is noisy that the model tends to overfit.
対象強調 It is the loss function that we aim to minimize.
結論強調 It is logistic regression that best fits linear boundary problems.
状態強調 It is during early stopping that the validation error remains low.
理由強調 It is because of the small batch size that the training is unstable.