Pythonista3 でAVAudioSourceNode を使って音を生成して鳴らそう！ 後編

この記事は、Pythonista3 Advent Calendar 2022 の16日目の記事です。

一方的な偏った目線で、Pythonista3 を紹介していきます。

ほぼ毎日iPhone（Pythonista3）で、コーディングをしている者です。よろしくお願いします。

以下、私の2022年12月時点の環境です。

sysInfo.log

--- SYSTEM INFORMATION ---
* Pythonista 3.3 (330025), Default interpreter 3.6.1
* iOS 16.1.1, model iPhone12,1, resolution (portrait) 828.0 x 1792.0 @ 2.0

他の環境(iPad や端末の種類、iOS のバージョン違い)では、意図としない挙動(エラーになる)なる場合もあります。ご了承ください。

ちなみに、model iPhone12,1 は、iPhone11 です。

前回までのあらすじ

• 音を出せた
  • objc_util モジュールでAVAudioEngine を中心に実装
    • AVAudioSourceNode
      • ObjCBlock で、音声データのルーティング
        • 数式を使ったsine波の生成
      • AudioBuffer, AudioBufferList 構造体の作成
    • connect によるNode の連結
• 音を止められた
  • ui モジュールでアプリライク & 終了指示の連携

まとめると、こんなにさっぱりするのですね！

今回は、波形の種類を変えたり、可視化したり、インタラクティブに操作できるようにしていきましょう！！

objc_util 色は弱めでPython （Pythonista3）の処理が中心です。

前回までのコード.py

from math import sin, pi
import ctypes

from objc_util import ObjCClass, ObjCBlock
import ui

import pdbg

CHANNEL = 1

OSStatus = ctypes.c_int32

AVAudioEngine = ObjCClass('AVAudioEngine')
AVAudioSourceNode = ObjCClass('AVAudioSourceNode')
AVAudioFormat = ObjCClass('AVAudioFormat')


class AudioBuffer(ctypes.Structure):
  _fields_ = [
    ('mNumberChannels', ctypes.c_uint32),
    ('mDataByteSize', ctypes.c_uint32),
    ('mData', ctypes.c_void_p),
  ]


class AudioBufferList(ctypes.Structure):
  _fields_ = [
    ('mNumberBuffers', ctypes.c_uint32),
    ('mBuffers', AudioBuffer * CHANNEL),
  ]


class Synth:
  def __init__(self):
    self.audioEngine: AVAudioEngine
    self.sampleRate: float = 44100.0  # set_up メソッド: outputNode より確定
    self.deltaTime: float = 0.0  # 1/sampleRate 時間間隔
    self.timex: float = 0.0  # Render の間隔カウンター

    self.render_block = ObjCBlock(
      self.source_node_render,
      restype=OSStatus,
      argtypes=[
        ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p,
        ctypes.POINTER(AudioBufferList)
      ])

    self.set_up()

  def set_up(self):
    audioEngine = AVAudioEngine.new()
    sourceNode = AVAudioSourceNode.alloc()
    mainMixer = audioEngine.mainMixerNode()
    outputNode = audioEngine.outputNode()
    format = outputNode.inputFormatForBus_(0)

    self.sampleRate = format.sampleRate()
    self.deltaTime = 1 / self.sampleRate

    inputFormat = AVAudioFormat.alloc(
    ).initWithCommonFormat_sampleRate_channels_interleaved_(
      format.commonFormat(), self.sampleRate, CHANNEL, format.isInterleaved())

    sourceNode.initWithFormat_renderBlock_(inputFormat, self.render_block)

    audioEngine.attachNode_(sourceNode)
    sourceNode.volume = 0.5

    audioEngine.connect_to_format_(sourceNode, mainMixer, inputFormat)
    audioEngine.connect_to_format_(mainMixer, outputNode, inputFormat)

    audioEngine.prepare()
    self.audioEngine = audioEngine

  def source_node_render(self,
                         _cmd: ctypes.c_void_p,
                         _isSilence_ptr: ctypes.c_void_p,
                         _timestamp_ptr: ctypes.c_void_p,
                         frameCount: ctypes.c_void_p,
                         outputData_ptr: ctypes.POINTER) -> OSStatus:
    ablPointer = outputData_ptr.contents
    for frame in range(frameCount):
      # 出力波形を変えていく
      sampleVal = sin(440.0 * 2.0 * pi * self.timex)
      self.timex += self.deltaTime

      for bufferr in range(ablPointer.mNumberBuffers):
        _mData = ablPointer.mBuffers[bufferr].mData
        _pointer = ctypes.POINTER(ctypes.c_float * frameCount)
        buffer = ctypes.cast(_mData, _pointer).contents
        buffer[frame] = sampleVal
    return 0

  def start(self):
    self.audioEngine.startAndReturnError_(None)

  def stop(self):
    self.audioEngine.stop()


class View(ui.View):
  def __init__(self, *args, **kwargs):
    ui.View.__init__(self, *args, **kwargs)
    self.synth = Synth()
    self.synth.start()

  def will_close(self):
    self.synth.stop()


if __name__ == '__main__':
  view = View()
  view.present()
  #view.present(style='fullscreen', orientations=['portrait'])

オシレーターをつくる

Oscillator（発振器・OSC）は、シンセサイザーで基本波形を作り出す機能です。

• 正弦波: sine
• 三角波: triangle
• ノコギリ波: sawtooth
• 矩形波: square

Building a Synthesizer in Swift. Making audio waveforms with… | by SwiftMoji | Better Programming

こちらを参考に作っていきます。

White Noise

おーっと、早速OSC で紹介していないのがでてきましたね😇

面白いですし、先々にやってしまった方がいいと思いまして、先に実装します。

whiteNoise.py

from random import uniform

# --- sine
#sampleVal = sin(440.0 * 2.0 * pi * self.timex)
# --- whiteNoise 
sampleVal = uniform(-1.0, 1.0)

random.random() ですと、0.0以上1.0未満 の抽出です。

音声データとして-1.0 〜 1.0 が欲しいので、random.uniform(-1.0, 1.0) で範囲を指定しています。

砂嵐の音になりましたね（現代人は、テレビやラジオのノイズ音とか知らないんだろうな）。

White Noise 以降は、関数化して作成していきます。

正弦波: sine

すでに、前回で呼び出し確認ができてるsine ですが、関数化をしていきます。

amplitude（振幅）とfrequency（周波数）を変数として事前に定義しておきます。

（振幅や周波数等の説明に関してはここでは割愛させていただきます）

# --- OSC
amplitude: float = 1.0
frequency: float = 440.0


def sine(time):
  wave = amplitude * sin(2.0 * pi * frequency * time)
  return wave

全体像:

sineを追加.py

from math import sin, pi
from random import uniform
import ctypes

from objc_util import ObjCClass, ObjCBlock
import ui

import pdbg

CHANNEL = 1

OSStatus = ctypes.c_int32

AVAudioEngine = ObjCClass('AVAudioEngine')
AVAudioSourceNode = ObjCClass('AVAudioSourceNode')
AVAudioFormat = ObjCClass('AVAudioFormat')


class AudioBuffer(ctypes.Structure):
  _fields_ = [
    ('mNumberChannels', ctypes.c_uint32),
    ('mDataByteSize', ctypes.c_uint32),
    ('mData', ctypes.c_void_p),
  ]


class AudioBufferList(ctypes.Structure):
  _fields_ = [
    ('mNumberBuffers', ctypes.c_uint32),
    ('mBuffers', AudioBuffer * CHANNEL),
  ]


# --- OSC
amplitude: float = 1.0
frequency: float = 440.0


def white_noise():
  return uniform(-1.0, 1.0)


def sine(time):
  wave = amplitude * sin(2.0 * pi * frequency * time)
  return wave


class Synth:
  def __init__(self):
    self.audioEngine: AVAudioEngine
    self.sampleRate: float = 44100.0  # set_up メソッド: outputNode より確定
    self.deltaTime: float = 0.0  # 1/sampleRate 時間間隔
    self.timex: float = 0.0  # Render の間隔カウンター

    self.render_block = ObjCBlock(
      self.source_node_render,
      restype=OSStatus,
      argtypes=[
        ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p,
        ctypes.POINTER(AudioBufferList)
      ])

    self.set_up()

  def set_up(self):
    audioEngine = AVAudioEngine.new()
    sourceNode = AVAudioSourceNode.alloc()
    mainMixer = audioEngine.mainMixerNode()
    outputNode = audioEngine.outputNode()
    format = outputNode.inputFormatForBus_(0)

    self.sampleRate = format.sampleRate()
    self.deltaTime = 1 / self.sampleRate

    inputFormat = AVAudioFormat.alloc(
    ).initWithCommonFormat_sampleRate_channels_interleaved_(
      format.commonFormat(), self.sampleRate, CHANNEL, format.isInterleaved())

    sourceNode.initWithFormat_renderBlock_(inputFormat, self.render_block)

    audioEngine.attachNode_(sourceNode)
    sourceNode.volume = 0.2

    audioEngine.connect_to_format_(sourceNode, mainMixer, inputFormat)
    audioEngine.connect_to_format_(mainMixer, outputNode, inputFormat)

    audioEngine.prepare()
    self.audioEngine = audioEngine

  def source_node_render(self,
                         _cmd: ctypes.c_void_p,
                         _isSilence_ptr: ctypes.c_void_p,
                         _timestamp_ptr: ctypes.c_void_p,
                         frameCount: ctypes.c_void_p,
                         outputData_ptr: ctypes.POINTER) -> OSStatus:
    ablPointer = outputData_ptr.contents
    for frame in range(frameCount):
      # 出力波形を変えていく
      # --- sine
      #sampleVal = sin(440.0 * 2.0 * pi * self.timex)
      # --- whiteNoise
      #sampleVal = uniform(-1.0, 1.0)
      #sampleVal = white_noise()
      # --- sine
      sampleVal = sine(self.timex)

      self.timex += self.deltaTime

      for bufferr in range(ablPointer.mNumberBuffers):
        _mData = ablPointer.mBuffers[bufferr].mData
        _pointer = ctypes.POINTER(ctypes.c_float * frameCount)
        buffer = ctypes.cast(_mData, _pointer).contents
        buffer[frame] = sampleVal
    return 0

  def start(self):
    self.audioEngine.startAndReturnError_(None)

  def stop(self):
    self.audioEngine.stop()


class View(ui.View):
  def __init__(self, *args, **kwargs):
    ui.View.__init__(self, *args, **kwargs)
    self.synth = Synth()
    self.synth.start()

  def will_close(self):
    self.synth.stop()


if __name__ == '__main__':
  view = View()
  view.present()
  #view.present(style='fullscreen', orientations=['portrait'])

他の波の関数は、sine の下に追加していきます。

さっくり関数のみ紹介していきます（この後class 化もしていくので面倒なら見てるだけでもいいかもしれません）。

三角波: triangle

def triangle(time):
  period = 1.0 / frequency
  currentTime = time % period
  value = currentTime / period
  result = 0.0
  if value < 0.25:
    result = value * 4
  elif value < 0.75:
    result = 2.0 - (value * 4.0)
  else:
    result = value * 4 - 4.0
  wave = amplitude * result
  return wave

ノコギリ波: sawtooth

def sawtooth(time):
  period = 1.0 / frequency
  currentTime = time % period
  wave = amplitude * ((currentTime / period) * 2 - 1.0)
  return wave

矩形波: square

def square(time):
  period = 1.0 / frequency
  currentTime = time % period
  if (currentTime / period) < 0.5:
    wave = amplitude
  else:
    wave = -1.0 * amplitude
  return wave

class 化と、インタラクティブな拡張性

関数実装.py

from math import sin, pi
from random import uniform
import ctypes

from objc_util import ObjCClass, ObjCBlock
import ui

import pdbg

CHANNEL = 1

OSStatus = ctypes.c_int32

AVAudioEngine = ObjCClass('AVAudioEngine')
AVAudioSourceNode = ObjCClass('AVAudioSourceNode')
AVAudioFormat = ObjCClass('AVAudioFormat')


class AudioBuffer(ctypes.Structure):
  _fields_ = [
    ('mNumberChannels', ctypes.c_uint32),
    ('mDataByteSize', ctypes.c_uint32),
    ('mData', ctypes.c_void_p),
  ]


class AudioBufferList(ctypes.Structure):
  _fields_ = [
    ('mNumberBuffers', ctypes.c_uint32),
    ('mBuffers', AudioBuffer * CHANNEL),
  ]


# --- OSC
amplitude: float = 1.0
frequency: float = 440.0


def white_noise():
  return uniform(-1.0, 1.0)


def sine(time):
  wave = amplitude * sin(2.0 * pi * frequency * time)
  return wave


def triangle(time):
  period = 1.0 / frequency
  currentTime = time % period
  value = currentTime / period
  result = 0.0
  if value < 0.25:
    result = value * 4
  elif value < 0.75:
    result = 2.0 - (value * 4.0)
  else:
    result = value * 4 - 4.0
  wave = amplitude * result
  return wave


def sawtooth(time):
  period = 1.0 / frequency
  currentTime = time % period
  wave = amplitude * ((currentTime / period) * 2 - 1.0)
  return wave


def square(time):
  period = 1.0 / frequency
  currentTime = time % period
  if (currentTime / period) < 0.5:
    wave = amplitude
  else:
    wave = -1.0 * amplitude
  return wave


class Synth:
  def __init__(self):
    self.audioEngine: AVAudioEngine
    self.sampleRate: float = 44100.0  # set_up メソッド: outputNode より確定
    self.deltaTime: float = 0.0  # 1/sampleRate 時間間隔
    self.timex: float = 0.0  # Render の間隔カウンター

    self.render_block = ObjCBlock(
      self.source_node_render,
      restype=OSStatus,
      argtypes=[
        ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p,
        ctypes.POINTER(AudioBufferList)
      ])

    self.set_up()

  def set_up(self):
    audioEngine = AVAudioEngine.new()
    sourceNode = AVAudioSourceNode.alloc()
    mainMixer = audioEngine.mainMixerNode()
    outputNode = audioEngine.outputNode()
    format = outputNode.inputFormatForBus_(0)

    self.sampleRate = format.sampleRate()
    self.deltaTime = 1 / self.sampleRate

    inputFormat = AVAudioFormat.alloc(
    ).initWithCommonFormat_sampleRate_channels_interleaved_(
      format.commonFormat(), self.sampleRate, CHANNEL, format.isInterleaved())

    sourceNode.initWithFormat_renderBlock_(inputFormat, self.render_block)

    audioEngine.attachNode_(sourceNode)
    sourceNode.volume = 0.2

    audioEngine.connect_to_format_(sourceNode, mainMixer, inputFormat)
    audioEngine.connect_to_format_(mainMixer, outputNode, inputFormat)

    audioEngine.prepare()
    self.audioEngine = audioEngine

  def source_node_render(self,
                         _cmd: ctypes.c_void_p,
                         _isSilence_ptr: ctypes.c_void_p,
                         _timestamp_ptr: ctypes.c_void_p,
                         frameCount: ctypes.c_void_p,
                         outputData_ptr: ctypes.POINTER) -> OSStatus:
    ablPointer = outputData_ptr.contents
    for frame in range(frameCount):
      # 出力波形を変えていく
      # --- whiteNoise
      #sampleVal = uniform(-1.0, 1.0)
      #sampleVal = white_noise()
      # --- sine
      #sampleVal = sin(440.0 * 2.0 * pi * self.timex)
      #sampleVal = sine(self.timex)
      # --- triangle
      #sampleVal = triangle(self.timex)
      # --- sawtooth
      #sampleVal = sawtooth(self.timex)
      # --- square
      sampleVal = square(self.timex)

      self.timex += self.deltaTime

      for bufferr in range(ablPointer.mNumberBuffers):
        _mData = ablPointer.mBuffers[bufferr].mData
        _pointer = ctypes.POINTER(ctypes.c_float * frameCount)
        buffer = ctypes.cast(_mData, _pointer).contents
        buffer[frame] = sampleVal
    return 0

  def start(self):
    self.audioEngine.startAndReturnError_(None)

  def stop(self):
    self.audioEngine.stop()


class View(ui.View):
  def __init__(self, *args, **kwargs):
    ui.View.__init__(self, *args, **kwargs)
    self.synth = Synth()
    self.synth.start()

  def will_close(self):
    self.synth.stop()


if __name__ == '__main__':
  view = View()
  view.present()
  #view.present(style='fullscreen', orientations=['portrait'])

一通り実装が完了しました。が、Oscillator としてclass 化した方が都合が良さそうですね。

いまはSynth のRender メソッド内で、毎回書き換え（コメントアウト）て、それぞれの波を選んでいるので、確認のイテレートも面倒です。

class Oscillator:

ひとまとめにしました。

self.wave_types の配列で波の関数をindex で呼び出せるようにしています。

また、white_noise メソッドは、引数が不要ですがメソッド全体と整える必要があるため、_ としています。

classの部分のみ.py

class Oscillator:
  def __init__(self):
    self.amplitude: float = 1.0
    self.frequency: float = 440.0
    self.wave_types = [
      self.sine,
      self.triangle,
      self.sawtooth,
      self.square,
      self.white_noise,
    ]

  def sine(self, time):
    wave = self.amplitude * sin(2.0 * pi * self.frequency * time)
    return wave

  def triangle(self, time):
    period = 1.0 / self.frequency
    currentTime = time % period
    value = currentTime / period
    result = 0.0
    if value < 0.25:
      result = value * 4
    elif value < 0.75:
      result = 2.0 - (value * 4.0)
    else:
      result = value * 4 - 4.0
    wave = self.amplitude * result
    return wave

  def sawtooth(self, time):
    period = 1.0 / self.frequency
    currentTime = time % period
    wave = self.amplitude * ((currentTime / period) * 2 - 1.0)
    return wave

  def square(self, time):
    period = 1.0 / self.frequency
    currentTime = time % period
    if (currentTime / period) < 0.5:
      wave = self.amplitude
    else:
      wave = -1.0 * self.amplitude
    return wave

  def white_noise(self, _):
    return uniform(-1.0, 1.0)

インタラクティブに

実行時に変化させるように、class View としているui.View の中に、スライダーで選択できるように組み込んでいきます。

osc 選択

def setup_type_slider(self):
  self.type_len = len(self.osc.wave_types) - 1
  self.type_osc = ui.Slider()
  self.type_osc.continuous = False
  self.type_osc.value = 0
  self.type_osc.flex = 'W'
  self.type_osc.action = self.change_osc
  self.add_subview(self.type_osc)
  
def change_osc(self, sender):
  val = int(sender.value * self.type_len)
  self.toneGenerator = self.osc.wave_types[val]
  self.type_osc.value = val / self.type_len
  print(self.toneGenerator)  # ← 今回の確認用

print を仕込んでいるので、ocs の切り替わりで現在の波を出力しています。

ui.Slider.continuous = False にて、（移動時ではなく）スライダーの位置がfix 時に数値を送り出しています。

val = int(sender.value * self.type_len) にて、配列index を指定。

self.type_osc.value = val / self.type_len にて、ui.Slider.value の位置調整をいています。

slider実装.py

from math import sin, pi
from random import uniform
import ctypes

from objc_util import ObjCClass, ObjCBlock
import ui

import pdbg

CHANNEL = 1

OSStatus = ctypes.c_int32

AVAudioEngine = ObjCClass('AVAudioEngine')
AVAudioSourceNode = ObjCClass('AVAudioSourceNode')
AVAudioFormat = ObjCClass('AVAudioFormat')


class AudioBuffer(ctypes.Structure):
  _fields_ = [
    ('mNumberChannels', ctypes.c_uint32),
    ('mDataByteSize', ctypes.c_uint32),
    ('mData', ctypes.c_void_p),
  ]


class AudioBufferList(ctypes.Structure):
  _fields_ = [
    ('mNumberBuffers', ctypes.c_uint32),
    ('mBuffers', AudioBuffer * CHANNEL),
  ]


# --- OSC
class Oscillator:
  def __init__(self):
    self.amplitude: float = 1.0
    self.frequency: float = 440.0
    self.wave_types = [
      self.sine,
      self.triangle,
      self.sawtooth,
      self.square,
      self.white_noise,
    ]

  def sine(self, time):
    wave = self.amplitude * sin(2.0 * pi * self.frequency * time)
    return wave

  def triangle(self, time):
    period = 1.0 / self.frequency
    currentTime = time % period
    value = currentTime / period
    result = 0.0
    if value < 0.25:
      result = value * 4
    elif value < 0.75:
      result = 2.0 - (value * 4.0)
    else:
      result = value * 4 - 4.0
    wave = self.amplitude * result
    return wave

  def sawtooth(self, time):
    period = 1.0 / self.frequency
    currentTime = time % period
    wave = self.amplitude * ((currentTime / period) * 2 - 1.0)
    return wave

  def square(self, time):
    period = 1.0 / self.frequency
    currentTime = time % period
    if (currentTime / period) < 0.5:
      wave = self.amplitude
    else:
      wave = -1.0 * self.amplitude
    return wave

  def white_noise(self, _):
    return uniform(-1.0, 1.0)


class Synth:
  def __init__(self, parent):
    self.parent: ui.View = parent
    self.audioEngine: AVAudioEngine
    self.sampleRate: float = 44100.0  # set_up メソッド: outputNode より確定
    self.deltaTime: float = 0.0  # 1/sampleRate 時間間隔
    self.timex: float = 0.0  # Render の間隔カウンター

    self.render_block = ObjCBlock(
      self.source_node_render,
      restype=OSStatus,
      argtypes=[
        ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p,
        ctypes.POINTER(AudioBufferList)
      ])

    self.set_up()

  def set_up(self):
    audioEngine = AVAudioEngine.new()
    sourceNode = AVAudioSourceNode.alloc()
    mainMixer = audioEngine.mainMixerNode()
    outputNode = audioEngine.outputNode()
    format = outputNode.inputFormatForBus_(0)

    self.sampleRate = format.sampleRate()
    self.deltaTime = 1 / self.sampleRate

    inputFormat = AVAudioFormat.alloc(
    ).initWithCommonFormat_sampleRate_channels_interleaved_(
      format.commonFormat(), self.sampleRate, CHANNEL, format.isInterleaved())

    sourceNode.initWithFormat_renderBlock_(inputFormat, self.render_block)

    audioEngine.attachNode_(sourceNode)
    sourceNode.volume = 0.1

    audioEngine.connect_to_format_(sourceNode, mainMixer, inputFormat)
    audioEngine.connect_to_format_(mainMixer, outputNode, inputFormat)

    audioEngine.prepare()
    self.audioEngine = audioEngine

  def source_node_render(self,
                         _cmd: ctypes.c_void_p,
                         _isSilence_ptr: ctypes.c_void_p,
                         _timestamp_ptr: ctypes.c_void_p,
                         frameCount: ctypes.c_void_p,
                         outputData_ptr: ctypes.POINTER) -> OSStatus:
    ablPointer = outputData_ptr.contents
    for frame in range(frameCount):
      sampleVal = self.parent.toneGenerator(self.timex)

      self.timex += self.deltaTime

      for bufferr in range(ablPointer.mNumberBuffers):
        _mData = ablPointer.mBuffers[bufferr].mData
        _pointer = ctypes.POINTER(ctypes.c_float * frameCount)
        buffer = ctypes.cast(_mData, _pointer).contents
        buffer[frame] = sampleVal
    return 0

  def start(self):
    self.audioEngine.startAndReturnError_(None)

  def stop(self):
    self.audioEngine.stop()


class View(ui.View):
  def __init__(self, *args, **kwargs):
    ui.View.__init__(self, *args, **kwargs)

    self.osc = Oscillator()
    self.setup_type_slider()
    self.toneGenerator = self.osc.wave_types[0]

    self.synth = Synth(self)
    self.synth.start()

  def setup_type_slider(self):
    self.type_len = len(self.osc.wave_types) - 1
    self.type_osc = ui.Slider()
    self.type_osc.continuous = False
    self.type_osc.value = 0
    self.type_osc.flex = 'W'
    self.type_osc.action = self.change_osc
    self.add_subview(self.type_osc)

  def change_osc(self, sender):
    val = int(sender.value * self.type_len)
    self.toneGenerator = self.osc.wave_types[val]
    self.type_osc.value = val / self.type_len
    print(self.toneGenerator)

  def will_close(self):
    self.synth.stop()


if __name__ == '__main__':
  view = View()
  view.present()
  #view.present(style='fullscreen', orientations=['portrait'])

frequency 選択

波形の変更と同様にfrequency（周波数）もスライダーで変更できるようにしましょう。

ほぼ同様の実装内容です。ui.Slider.continuous をデフォルトのTrue としており、スライダー位置が動く度に数値が変化します。

self.max_level_frq = 880.0 と最大周波数を880.0 と決め打ちにし、ui.Slider.value の0.0 〜 1.0 の間のvalue 調整をしています。

frequencyを追加.py

from math import sin, pi
from random import uniform
import ctypes

from objc_util import ObjCClass, ObjCBlock
import ui

import pdbg

CHANNEL = 1

OSStatus = ctypes.c_int32

AVAudioEngine = ObjCClass('AVAudioEngine')
AVAudioSourceNode = ObjCClass('AVAudioSourceNode')
AVAudioFormat = ObjCClass('AVAudioFormat')


class AudioBuffer(ctypes.Structure):
  _fields_ = [
    ('mNumberChannels', ctypes.c_uint32),
    ('mDataByteSize', ctypes.c_uint32),
    ('mData', ctypes.c_void_p),
  ]


class AudioBufferList(ctypes.Structure):
  _fields_ = [
    ('mNumberBuffers', ctypes.c_uint32),
    ('mBuffers', AudioBuffer * CHANNEL),
  ]


# --- OSC
class Oscillator:
  def __init__(self):
    self.amplitude: float = 1.0
    self.frequency: float = 440.0
    self.wave_types = [
      self.sine,
      self.triangle,
      self.sawtooth,
      self.square,
      self.white_noise,
    ]

  def sine(self, time):
    wave = self.amplitude * sin(2.0 * pi * self.frequency * time)
    return wave

  def triangle(self, time):
    period = 1.0 / self.frequency
    currentTime = time % period
    value = currentTime / period
    result = 0.0
    if value < 0.25:
      result = value * 4
    elif value < 0.75:
      result = 2.0 - (value * 4.0)
    else:
      result = value * 4 - 4.0
    wave = self.amplitude * result
    return wave

  def sawtooth(self, time):
    period = 1.0 / self.frequency
    currentTime = time % period
    wave = self.amplitude * ((currentTime / period) * 2 - 1.0)
    return wave

  def square(self, time):
    period = 1.0 / self.frequency
    currentTime = time % period
    if (currentTime / period) < 0.5:
      wave = self.amplitude
    else:
      wave = -1.0 * self.amplitude
    return wave

  def white_noise(self, _):
    return uniform(-1.0, 1.0)


class Synth:
  def __init__(self, parent):
    self.parent: ui.View = parent
    self.audioEngine: AVAudioEngine
    self.sampleRate: float = 44100.0  # set_up メソッド: outputNode より確定
    self.deltaTime: float = 0.0  # 1/sampleRate 時間間隔
    self.timex: float = 0.0  # Render の間隔カウンター

    self.render_block = ObjCBlock(
      self.source_node_render,
      restype=OSStatus,
      argtypes=[
        ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p,
        ctypes.POINTER(AudioBufferList)
      ])

    self.set_up()

  def set_up(self):
    audioEngine = AVAudioEngine.new()
    sourceNode = AVAudioSourceNode.alloc()
    mainMixer = audioEngine.mainMixerNode()
    outputNode = audioEngine.outputNode()
    format = outputNode.inputFormatForBus_(0)

    self.sampleRate = format.sampleRate()
    self.deltaTime = 1 / self.sampleRate

    inputFormat = AVAudioFormat.alloc(
    ).initWithCommonFormat_sampleRate_channels_interleaved_(
      format.commonFormat(), self.sampleRate, CHANNEL, format.isInterleaved())

    sourceNode.initWithFormat_renderBlock_(inputFormat, self.render_block)

    audioEngine.attachNode_(sourceNode)
    sourceNode.volume = 0.1

    audioEngine.connect_to_format_(sourceNode, mainMixer, inputFormat)
    audioEngine.connect_to_format_(mainMixer, outputNode, inputFormat)

    audioEngine.prepare()
    self.audioEngine = audioEngine

  def source_node_render(self,
                         _cmd: ctypes.c_void_p,
                         _isSilence_ptr: ctypes.c_void_p,
                         _timestamp_ptr: ctypes.c_void_p,
                         frameCount: ctypes.c_void_p,
                         outputData_ptr: ctypes.POINTER) -> OSStatus:
    ablPointer = outputData_ptr.contents
    for frame in range(frameCount):
      sampleVal = self.parent.toneGenerator(self.timex)

      self.timex += self.deltaTime

      for bufferr in range(ablPointer.mNumberBuffers):
        _mData = ablPointer.mBuffers[bufferr].mData
        _pointer = ctypes.POINTER(ctypes.c_float * frameCount)
        buffer = ctypes.cast(_mData, _pointer).contents
        buffer[frame] = sampleVal
    return 0

  def start(self):
    self.audioEngine.startAndReturnError_(None)

  def stop(self):
    self.audioEngine.stop()


class View(ui.View):
  def __init__(self, *args, **kwargs):
    ui.View.__init__(self, *args, **kwargs)

    self.setup_osc()

    self.synth = Synth(self)
    self.synth.start()

  def setup_osc(self):
    self.osc = Oscillator()
    self.setup_type_slider()
    self.setup_frq_slider()
    self.toneGenerator = self.osc.wave_types[0]

  def setup_type_slider(self):
    self.type_len = len(self.osc.wave_types) - 1
    self.type_osc = ui.Slider()
    self.type_osc.continuous = False
    self.type_osc.value = 0
    self.type_osc.flex = 'W'
    self.type_osc.action = self.change_osc
    self.add_subview(self.type_osc)

  def change_osc(self, sender):
    val = int(sender.value * self.type_len)
    self.toneGenerator = self.osc.wave_types[val]
    self.type_osc.value = val / self.type_len

  def setup_frq_slider(self):
    self.max_level_frq = 880.0
    self.level_frq = ui.Slider()
    self.level_frq.value = self.osc.frequency / self.max_level_frq
    self.level_frq.flex = 'W'
    self.level_frq.action = self.change_frq
    self.add_subview(self.level_frq)

  def change_frq(self, sender):
    val = sender.value * self.max_level_frq
    self.osc.frequency = val

  def layout(self):
    self.level_frq.y = self.type_osc.height

  def will_close(self):
    self.synth.stop()


if __name__ == '__main__':
  view = View()
  #view.present()
  # スライダー操作時にView が動いてしまうため
  view.present(style='fullscreen', orientations=['portrait'])

音声情報の可視化

ここまでくると、View 上部にスライダー2本のみ。という見た目が寂しくなってきました。

（GIF でも、同じ絵で押し通せなくなっています）

AVAudioEngineを使って音声にリアルタイムにエフェクトをかけて保存する方法(サンプルあり) - Qiita

render として自作したblock（source_node_render）の音になる情報を、別のblock を作ることで取得ができそうです。

installTapOnBus:bufferSize:format:block: | Apple Developer Documentation

tap_block として、ObjCBlock を定義します:

self.tap_block = ObjCBlock(
  self.audio_node_tap,
  restype=None,
  argtypes=[
    ctypes.c_void_p,
    ctypes.c_void_p,
    ctypes.c_void_p,
  ])

コールバック関数として、audio_node_tap をつくります:

def audio_node_tap(self, _cmd, buffer, when):
  # objc_util の処理
  buf = ObjCInstance(buffer)
  _array = buf.floatChannelData()[0]
  
  # ここからPython の処理
  # 画像生成用に配列組み替え
  np_buff = np.ctypeslib.as_array(_array, (256, 16))
  with BytesIO() as bIO:
    matplotlib.image.imsave(bIO, np_buff + 1, format='png')
    img = ui.Image.from_data(bIO.getvalue())
    self.parent.visualize_view.image = img

objc_util で考えることと、Python（Pythonista3）で考えることを分けて、混乱する要因を減らします。

先にobjc_util の処理を進めます。

objc_util の処理

Synth 内の、connect 処理をしている最終点の mainMixer から取得をします。

（数式で生成した情報を取りたい場合にはsourceNode）

audioEngine.connect_to_format_(mainMixer, outputNode, inputFormat)
audioEngine.prepare()

_bufsize = 64 * 64  # 取得する情報量
mainMixer.installTapOnBus_bufferSize_format_block_(
  0, _bufsize, inputFormat, self.tap_block)
self.audioEngine = audioEngine

_bufsize にて、取りたいサイズ指定をします。無駄に多くても処理が追いつかない可能性があるので、4096 = 64*64 としました。

installTapOnBus_bufferSize_format_block_ にて、audio_node_tap が呼び出されるようになりました。

def audio_node_tap(self, _cmd, buffer, when):
  # objc_util の処理
  buf = ObjCInstance(buffer)
  _array = buf.floatChannelData()[0]

型定義を事細かく説明していましたが、objc_util.ObjCInstance で問題ないのであればctypes.c_void_p で定義してしまっています。

ObjCInstance で問題ないなら、それはそれでオッケー的な考え方です（雑）。

floatChannelData() は、配列で来るのでindexの[0] で情報を貰います。

block は、毎回すごい早さで呼び出されるので、print 確認が大変です。

実行後に即終了させてprint 出力量を減らし、必要な情報を確認する作業をしています（もちろん事前にDocumentation で返ってくる内容を把握することも大事です）。

音データの内部構造など、めちゃくちゃ面白く深いのですが、長くなりそうなので泣く泣く割愛します。。。

Python の処理

audio_node_tap メソッドにて、Objective-C（objc_util）の情報を、通常のPython で処理できるようになりました。

ざっくりとした説明ですと、PCM フォーマットの音の情報（-1.0 〜 1.0）が配列として入っています。

最終的にui.Image へ乗せたいので、バイト列の画像情報に変換をさせたいのです。

ここからのPython 処理
当時どのように調べ実装したのかの記録がなく
ゆるふわな、説明になってしまいます🙇

NumPy で配列を整える

配列の形状を変更させるのは、NumPy が強いので（しかもctypes 経由の操作なので）、np.ctypeslib.as_array で形状を整えます。

最終出力画像を確認しながらの調整ですが、4096 = (256, 16) が綺麗な横並びとなったので、そのような数値としています。

Matplotlib で画像を作る

Matplotlib でグラフは作りません。

（かつて）音データだった配列を、バイトデータとして画像化します。

np_buff = np.ctypeslib.as_array(_array, (256, 16))
with BytesIO() as bIO:
  matplotlib.image.imsave(bIO, np_buff + 1, format='png')
  img = ui.Image.from_data(bIO.getvalue())

PIL モジュールでもいい気がしますが、なぜかMatplotlib を使っている理由は不明です。。。

Python Imaging Library — Python 3.6.1 documentation

Matplotlib の方が当時やりやすかったのかもしれません。

使いやすい、やりやすい方法で実装いただいて問題ありません。

変数img として、ui.Image に乗せられる状態になりました。

ui.View での可視化

最終出力となる音データを抜き出し、画像情報に変わるよう配列を整形しui.Image としてimg の準備まで完了しました。

class のView へ、乗せたいものを乗せていきましょう:

from math import sin, pi
from random import uniform
import ctypes
from io import BytesIO

import numpy as np
import matplotlib.image

from objc_util import ObjCClass, ObjCInstance, ObjCBlock
import ui

import pdbg

CHANNEL = 1

OSStatus = ctypes.c_int32

AVAudioEngine = ObjCClass('AVAudioEngine')
AVAudioSourceNode = ObjCClass('AVAudioSourceNode')
AVAudioFormat = ObjCClass('AVAudioFormat')


class AudioBuffer(ctypes.Structure):
  _fields_ = [
    ('mNumberChannels', ctypes.c_uint32),
    ('mDataByteSize', ctypes.c_uint32),
    ('mData', ctypes.c_void_p),
  ]


class AudioBufferList(ctypes.Structure):
  _fields_ = [
    ('mNumberBuffers', ctypes.c_uint32),
    ('mBuffers', AudioBuffer * CHANNEL),
  ]


class Oscillator:
  def __init__(self):
    self.amplitude: float = 1.0
    self.frequency: float = 440.0
    self.wave_types = [
      self.sine,
      self.triangle,
      self.sawtooth,
      self.square,
      self.white_noise,
    ]

  def sine(self, time):
    wave = self.amplitude * sin(2.0 * pi * self.frequency * time)
    return wave

  def triangle(self, time):
    period = 1.0 / self.frequency
    currentTime = time % period
    value = currentTime / period
    result = 0.0
    if value < 0.25:
      result = value * 4
    elif value < 0.75:
      result = 2.0 - (value * 4.0)
    else:
      result = value * 4 - 4.0
    wave = self.amplitude * result
    return wave

  def sawtooth(self, time):
    period = 1.0 / self.frequency
    currentTime = time % period
    wave = self.amplitude * ((currentTime / period) * 2 - 1.0)
    return wave

  def square(self, time):
    period = 1.0 / self.frequency
    currentTime = time % period
    if (currentTime / period) < 0.5:
      wave = self.amplitude
    else:
      wave = -1.0 * self.amplitude
    return wave

  def white_noise(self, _):
    return uniform(-1.0, 1.0)


class Synth:
  def __init__(self, parent):
    self.parent: ui.View = parent  # 親となるui.View
    self.audioEngine: AVAudioEngine
    self.sampleRate: float = 44100.0  # set_up メソッド: outputNode より確定
    self.deltaTime: float = 0.0  # 1/sampleRate 時間間隔
    self.timex: float = 0.0  # Render の間隔カウンター

    self.render_block = ObjCBlock(
      self.source_node_render,
      restype=OSStatus,
      argtypes=[
        ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p,
        ctypes.POINTER(AudioBufferList)
      ])

    self.tap_block = ObjCBlock(
      self.audio_node_tap,
      restype=None,
      argtypes=[
        ctypes.c_void_p,
        ctypes.c_void_p,
        ctypes.c_void_p,
      ])

    self.set_up()

  def set_up(self):
    audioEngine = AVAudioEngine.new()
    sourceNode = AVAudioSourceNode.alloc()
    mainMixer = audioEngine.mainMixerNode()
    outputNode = audioEngine.outputNode()
    format = outputNode.inputFormatForBus_(0)

    self.sampleRate = format.sampleRate()
    self.deltaTime = 1 / self.sampleRate

    inputFormat = AVAudioFormat.alloc(
    ).initWithCommonFormat_sampleRate_channels_interleaved_(
      format.commonFormat(), self.sampleRate, CHANNEL, format.isInterleaved())

    sourceNode.initWithFormat_renderBlock_(inputFormat, self.render_block)

    audioEngine.attachNode_(sourceNode)
    sourceNode.volume = 0.1

    audioEngine.connect_to_format_(sourceNode, mainMixer, inputFormat)
    audioEngine.connect_to_format_(mainMixer, outputNode, inputFormat)

    audioEngine.prepare()

    _bufsize = 64 * 64  # 取得する情報量
    mainMixer.installTapOnBus_bufferSize_format_block_(
      0, _bufsize, inputFormat, self.tap_block)
    self.audioEngine = audioEngine

  def source_node_render(self,
                         _cmd: ctypes.c_void_p,
                         _isSilence_ptr: ctypes.c_void_p,
                         _timestamp_ptr: ctypes.c_void_p,
                         frameCount: ctypes.c_void_p,
                         outputData_ptr: ctypes.POINTER) -> OSStatus:
    ablPointer = outputData_ptr.contents
    for frame in range(frameCount):
      sampleVal = self.parent.toneGenerator(self.timex)

      self.timex += self.deltaTime

      for bufferr in range(ablPointer.mNumberBuffers):
        _mData = ablPointer.mBuffers[bufferr].mData
        _pointer = ctypes.POINTER(ctypes.c_float * frameCount)
        buffer = ctypes.cast(_mData, _pointer).contents
        buffer[frame] = sampleVal
    return 0

  def audio_node_tap(self, _cmd, buffer, when):
    buf = ObjCInstance(buffer)
    _array = buf.floatChannelData()[0]
    # 画像生成用に配列組み替え
    np_buff = np.ctypeslib.as_array(_array, (256, 16))
    with BytesIO() as bIO:
      matplotlib.image.imsave(bIO, np_buff + 1, format='png')
      img = ui.Image.from_data(bIO.getvalue())
      self.parent.visualize_view.image = img

  def start(self):
    self.audioEngine.startAndReturnError_(None)

  def stop(self):
    self.audioEngine.stop()


class View(ui.View):
  def __init__(self, *args, **kwargs):
    ui.View.__init__(self, *args, **kwargs)

    self.visualize_view = ui.ImageView()
    self.visualize_view.bg_color = 0
    self.visualize_view.flex = 'WH'
    self.add_subview(self.visualize_view)

    self.type_osc = ui.Slider()
    self.level_frq = ui.Slider()
    self.osc_log = ui.Label()
    self.frq_log = ui.Label()

    self.osc = Oscillator()
    self.toneGenerator: Oscillator
    self.setup_osc()

    self.synth = Synth(self)
    self.synth.start()

  def setup_osc(self):
    self.toneGenerator = self.osc.wave_types[0]
    self.setup_type_osc()
    self.setup_frq_level()

  def setup_type_osc(self):
    # --- slider
    self.type_len = len(self.osc.wave_types) - 1
    self.type_osc.continuous = False
    self.type_osc.value = 0
    self.type_osc.flex = 'W'
    self.type_osc.action = self.change_osc
    self.add_subview(self.type_osc)

    # --- label
    self.osc_log.text = self.toneGenerator.__name__
    self.osc_log.bg_color = 1
    self.osc_log.flex = 'W'
    self.osc_log.size_to_fit()
    self.add_subview(self.osc_log)

  def change_osc(self, sender):
    val = int(sender.value * self.type_len)
    self.toneGenerator = self.osc.wave_types[val]
    self.type_osc.value = val / self.type_len
    self.osc_log.text = self.toneGenerator.__name__

  def setup_frq_level(self):
    # --- slider
    self.max_level_frq = 880.0
    self.level_frq.value = self.osc.frequency / self.max_level_frq
    self.level_frq.flex = 'W'
    self.level_frq.action = self.change_frq
    self.add_subview(self.level_frq)

    # --- label
    self.frq_log.text = f'{self.osc.frequency}'
    self.frq_log.bg_color = 1
    self.frq_log.flex = 'W'
    self.frq_log.size_to_fit()
    self.add_subview(self.frq_log)

  def change_frq(self, sender):
    val = sender.value * self.max_level_frq
    self.osc.frequency = val
    self.frq_log.text = f'{self.osc.frequency}'

  def layout(self):
    # --- slider
    self.type_osc.y = self.type_osc.height
    self.level_frq.y = self.type_osc.height + self.type_osc.y * 2

    # --- label
    self.osc_log.y = self.frame.height / 2 - self.osc_log.height
    self.frq_log.y = self.osc_log.y + self.frq_log.height

    logs_width = self.frame.width / 2
    self.osc_log.width = self.frq_log.width = logs_width

  def will_close(self):
    self.synth.stop()


if __name__ == '__main__':
  view = View()
  #view.present()
  # スライダー操作時にView が動いてしまうため
  view.present(style='fullscreen', orientations=['portrait'])

かなりfat なView class となってしまいました、、、

• 音声情報の可視化
  • self.visualize_view
• osc 関係
  • 操作
    • self.type_osc
  • 表示
    • self.osc_log
• 周波数（frequency）関係
  • 操作
    • self.level_frq
  • 表示
    • self.frq_log

View の画像の柄により、波形の特徴がわかります。

赤 → 黄 → 緑 → 青

の、色の移り変わりで数値の強弱が出力されているとイメージするとわかりやすいです。

正弦波: sine

綺麗に波打つ波形になるので、グラデーションの変化が滑らかに出ています。

三角波: triangle

下から上へ直線に△ 勾配している状態です。

ノコギリ波: sawtooth

|\ の勾配となるので、青 から突然赤 になっているのが確認できます。

矩形波: square

_| ̄|_ と、上下間を直角に変化させるので、青 と赤 しか出ていません。

ノイズ: WhiteNoise

-1.0 〜 1.0 をランダムで変化させているので、統一性がなくカラフルに出力されています。

作曲編曲！？波形をいじってみよう

波形同士を組み合わせることで、定型の波形から変化させることができます。

Oscillator class の中身を一部改変して、独自の波形を音として出力してみましょう。

変化がキツい三角波と、波形を混ぜ合わせるメソッドをつくりました:

def tone_triangle(self, time, frq=None):
  frequency = frq if frq else self.frequency
  period = 1.0 / frequency
  currentTime = time % period
  value = currentTime / period
  result = 0.0
  if value < 0.0:
    result = value * 4
  elif value > 0.8:
    result = value * 4 - 4.0
  else:
    result = 0
  wave = self.amplitude * result
  return wave

def mixwave(self, time):
  _step = 3 + int(sin(pi * time) * 10)
  steps = _step if _step else 1
  wave01 = self.square(time) * self.tone_triangle(time, steps)
  wave02 = self.white_noise(time) * self.tone_triangle(time, 1)
  wave = wave01 + wave02
  return wave

引数にfrq=None を付けて、self.frequency 以外でもfrequency を設定できるようにしています:

def tone_triangle(self, time, frq=None):
  frequency = frq if frq else self.frequency

frq=None は他の関数にも、実装させています。

Oscillator の配列にも登録してあげましょう:

self.wave_types = [
  self.mixwave,  # new!
  self.sine,
  self.triangle,
  self.sawtooth,
  self.square,
  self.white_noise,
  self.tone_triangle,  # new!
]

コード例では、画面の色変化のバキバキが強いので、小さくしています。

from math import sin, pi
from random import uniform
import ctypes
from io import BytesIO

import numpy as np
import matplotlib.image

from objc_util import ObjCClass, ObjCInstance, ObjCBlock
import ui

import pdbg

CHANNEL = 1

OSStatus = ctypes.c_int32

AVAudioEngine = ObjCClass('AVAudioEngine')
AVAudioSourceNode = ObjCClass('AVAudioSourceNode')
AVAudioFormat = ObjCClass('AVAudioFormat')


class AudioBuffer(ctypes.Structure):
  _fields_ = [
    ('mNumberChannels', ctypes.c_uint32),
    ('mDataByteSize', ctypes.c_uint32),
    ('mData', ctypes.c_void_p),
  ]


class AudioBufferList(ctypes.Structure):
  _fields_ = [
    ('mNumberBuffers', ctypes.c_uint32),
    ('mBuffers', AudioBuffer * CHANNEL),
  ]


class Oscillator:
  def __init__(self):
    self.amplitude: float = 1.0
    self.frequency: float = 440.0
    self.wave_types = [
      self.mixwave,
      self.sine,
      self.triangle,
      self.sawtooth,
      self.square,
      self.white_noise,
      self.tone_triangle,
    ]

  def sine(self, time, *args):
    frequency = args[0] if args else self.frequency
    wave = self.amplitude * sin(2.0 * pi * frequency * time)
    return wave

  def triangle(self, time, frq=None):
    frequency = frq if frq else self.frequency
    period = 1.0 / frequency
    currentTime = time % period
    value = currentTime / period
    result = 0.0
    if value < 0.25:
      result = value * 4
    elif value < 0.75:
      result = 2.0 - (value * 4.0)
    else:
      result = value * 4 - 4.0
    wave = self.amplitude * result
    return wave

  def sawtooth(self, time, frq=None):
    frequency = frq if frq else self.frequency
    period = 1.0 / frequency
    currentTime = time % period
    wave = self.amplitude * ((currentTime / period) * 2 - 1.0)
    return wave

  def square(self, time, frq=None):
    frequency = frq if frq else self.frequency
    period = 1.0 / frequency
    currentTime = time % period
    if (currentTime / period) < 0.5:
      wave = self.amplitude
    else:
      wave = -1.0 * self.amplitude
    return wave

  def white_noise(self, _, frq=None):
    return uniform(-1.0, 1.0)

  def tone_triangle(self, time, frq=None):
    frequency = frq if frq else self.frequency
    period = 1.0 / frequency
    currentTime = time % period
    value = currentTime / period
    result = 0.0
    if value < 0.0:
      result = value * 4
    elif value > 0.8:
      result = value * 4 - 4.0
    else:
      result = 0
    wave = self.amplitude * result
    return wave

  def mixwave(self, time):
    _step = 3 + int(sin(pi * time) * 10)
    steps = _step if _step else 1
    wave01 = self.square(time) * self.tone_triangle(time, steps)
    wave02 = self.white_noise(time) * self.tone_triangle(time, 1)
    wave = wave01 + wave02
    return wave


class Synth:
  def __init__(self, parent):
    self.parent: ui.View = parent  # 親となるui.View
    self.audioEngine: AVAudioEngine
    self.sampleRate: float = 44100.0  # set_up メソッド: outputNode より確定
    self.deltaTime: float = 0.0  # 1/sampleRate 時間間隔
    self.timex: float = 0.0  # Render の間隔カウンター

    self.render_block = ObjCBlock(
      self.source_node_render,
      restype=OSStatus,
      argtypes=[
        ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p,
        ctypes.POINTER(AudioBufferList)
      ])

    self.tap_block = ObjCBlock(
      self.audio_node_tap,
      restype=None,
      argtypes=[
        ctypes.c_void_p,
        ctypes.c_void_p,
        ctypes.c_void_p,
      ])

    self.set_up()

  def set_up(self):
    audioEngine = AVAudioEngine.new()
    sourceNode = AVAudioSourceNode.alloc()
    mainMixer = audioEngine.mainMixerNode()
    outputNode = audioEngine.outputNode()
    format = outputNode.inputFormatForBus_(0)

    self.sampleRate = format.sampleRate()
    self.deltaTime = 1 / self.sampleRate

    inputFormat = AVAudioFormat.alloc(
    ).initWithCommonFormat_sampleRate_channels_interleaved_(
      format.commonFormat(), self.sampleRate, CHANNEL, format.isInterleaved())

    sourceNode.initWithFormat_renderBlock_(inputFormat, self.render_block)

    audioEngine.attachNode_(sourceNode)
    sourceNode.volume = 0.1

    audioEngine.connect_to_format_(sourceNode, mainMixer, inputFormat)
    audioEngine.connect_to_format_(mainMixer, outputNode, inputFormat)

    audioEngine.prepare()

    _bufsize = 64 * 64  # 取得する情報量
    mainMixer.installTapOnBus_bufferSize_format_block_(
      0, _bufsize, inputFormat, self.tap_block)
    self.audioEngine = audioEngine

  def source_node_render(self,
                         _cmd: ctypes.c_void_p,
                         _isSilence_ptr: ctypes.c_void_p,
                         _timestamp_ptr: ctypes.c_void_p,
                         frameCount: ctypes.c_void_p,
                         outputData_ptr: ctypes.POINTER) -> OSStatus:
    ablPointer = outputData_ptr.contents
    for frame in range(frameCount):
      sampleVal = self.parent.toneGenerator(self.timex)

      self.timex += self.deltaTime

      for bufferr in range(ablPointer.mNumberBuffers):
        _mData = ablPointer.mBuffers[bufferr].mData
        _pointer = ctypes.POINTER(ctypes.c_float * frameCount)
        buffer = ctypes.cast(_mData, _pointer).contents
        buffer[frame] = sampleVal
    return 0

  def audio_node_tap(self, _cmd, buffer, when):
    buf = ObjCInstance(buffer)
    _array = buf.floatChannelData()[0]
    # 画像生成用に配列組み替え
    np_buff = np.ctypeslib.as_array(_array, (256, 16))
    with BytesIO() as bIO:
      matplotlib.image.imsave(bIO, np_buff + 1, format='png')
      img = ui.Image.from_data(bIO.getvalue())
      self.parent.visualize_view.image = img

  def start(self):
    self.audioEngine.startAndReturnError_(None)

  def stop(self):
    self.audioEngine.stop()


class View(ui.View):
  def __init__(self, *args, **kwargs):
    ui.View.__init__(self, *args, **kwargs)

    self.visualize_view = ui.ImageView()
    self.visualize_view.bg_color = 0
    #self.visualize_view.flex = 'WH'
    self.visualize_view.flex = 'TBRL'
    self.add_subview(self.visualize_view)

    self.type_osc = ui.Slider()
    self.level_frq = ui.Slider()
    self.osc_log = ui.Label()
    self.frq_log = ui.Label()

    self.osc = Oscillator()
    self.toneGenerator: Oscillator
    self.setup_osc()

    self.synth = Synth(self)
    self.synth.start()

  def setup_osc(self):
    self.toneGenerator = self.osc.wave_types[0]
    self.setup_type_osc()
    self.setup_frq_level()

  def setup_type_osc(self):
    # --- slider
    self.type_len = len(self.osc.wave_types) - 1
    self.type_osc.continuous = False
    self.type_osc.value = 0
    self.type_osc.flex = 'W'
    self.type_osc.action = self.change_osc
    self.add_subview(self.type_osc)

    # --- label
    self.osc_log.text = self.toneGenerator.__name__
    self.osc_log.bg_color = 1
    self.osc_log.flex = 'W'
    self.osc_log.size_to_fit()
    self.add_subview(self.osc_log)

  def change_osc(self, sender):
    val = int(sender.value * self.type_len)
    self.toneGenerator = self.osc.wave_types[val]
    self.type_osc.value = val / self.type_len
    self.osc_log.text = self.toneGenerator.__name__

  def setup_frq_level(self):
    # --- slider
    self.max_level_frq = 880.0
    self.level_frq.value = self.osc.frequency / self.max_level_frq
    self.level_frq.flex = 'W'
    self.level_frq.action = self.change_frq
    self.add_subview(self.level_frq)

    # --- label
    self.frq_log.text = f'{self.osc.frequency}'
    self.frq_log.bg_color = 1
    self.frq_log.flex = 'W'
    self.frq_log.size_to_fit()
    self.add_subview(self.frq_log)

  def change_frq(self, sender):
    val = sender.value * self.max_level_frq
    self.osc.frequency = val
    self.frq_log.text = f'{self.osc.frequency}'

  def layout(self):
    # --- slider
    self.type_osc.y = self.type_osc.height
    self.level_frq.y = self.type_osc.height + self.type_osc.y * 2

    # --- label
    self.osc_log.y = self.frame.height / 2 - self.osc_log.height
    self.frq_log.y = self.osc_log.y + self.frq_log.height

    logs_width = self.frame.width / 2
    self.osc_log.width = self.frq_log.width = logs_width

  def will_close(self):
    self.synth.stop()


if __name__ == '__main__':
  view = View()
  #view.present()
  # スライダー操作時にView が動いてしまうため
  view.present(style='fullscreen', orientations=['portrait'])

エフェクトをつけよう

AVAudioUnitDelay | Apple Developer Documentation

AVAudioUnitのパラメータ詳細（基本編） - Qiita

AudioUnit のエフェクトが使えます！

呼び出しは簡単で:

AVAudioUnitDelay = ObjCClass('AVAudioUnitDelay')

delay = AVAudioUnitDelay.new()
delay.delayTime = 0.3
delay.feedback = 80

アタッチして、Node にconnect すれば、反映されます:

audioEngine.attachNode_(delay)
    
audioEngine.connect_to_format_(sourceNode, delay, inputFormat)
audioEngine.connect_to_format_(delay, mainMixer, inputFormat)

音がカオスなので、エフェクトがわかりにくいかもしれませんが、無事にかかっていますね！

osc の切り替えや周波数変更時が、わかりやすいかもしれません。

from math import sin, pi
from random import uniform
import ctypes
from io import BytesIO

import numpy as np
import matplotlib.image

from objc_util import ObjCClass, ObjCInstance, ObjCBlock
import ui

import pdbg

CHANNEL = 1

OSStatus = ctypes.c_int32

AVAudioEngine = ObjCClass('AVAudioEngine')
AVAudioSourceNode = ObjCClass('AVAudioSourceNode')
AVAudioFormat = ObjCClass('AVAudioFormat')
AVAudioUnitDelay = ObjCClass('AVAudioUnitDelay')


class AudioBuffer(ctypes.Structure):
  _fields_ = [
    ('mNumberChannels', ctypes.c_uint32),
    ('mDataByteSize', ctypes.c_uint32),
    ('mData', ctypes.c_void_p),
  ]


class AudioBufferList(ctypes.Structure):
  _fields_ = [
    ('mNumberBuffers', ctypes.c_uint32),
    ('mBuffers', AudioBuffer * CHANNEL),
  ]


class Oscillator:
  def __init__(self):
    self.amplitude: float = 1.0
    self.frequency: float = 440.0
    self.wave_types = [
      self.mixwave,
      self.sine,
      self.triangle,
      self.sawtooth,
      self.square,
      self.white_noise,
      self.tone_triangle,
    ]

  def sine(self, time, *args):
    frequency = args[0] if args else self.frequency
    wave = self.amplitude * sin(2.0 * pi * frequency * time)
    return wave

  def triangle(self, time, frq=None):
    frequency = frq if frq else self.frequency
    period = 1.0 / frequency
    currentTime = time % period
    value = currentTime / period
    result = 0.0
    if value < 0.25:
      result = value * 4
    elif value < 0.75:
      result = 2.0 - (value * 4.0)
    else:
      result = value * 4 - 4.0
    wave = self.amplitude * result
    return wave

  def sawtooth(self, time, frq=None):
    frequency = frq if frq else self.frequency
    period = 1.0 / frequency
    currentTime = time % period
    wave = self.amplitude * ((currentTime / period) * 2 - 1.0)
    return wave

  def square(self, time, frq=None):
    frequency = frq if frq else self.frequency
    period = 1.0 / frequency
    currentTime = time % period
    if (currentTime / period) < 0.5:
      wave = self.amplitude
    else:
      wave = -1.0 * self.amplitude
    return wave

  def white_noise(self, _, frq=None):
    return uniform(-1.0, 1.0)

  def tone_triangle(self, time, frq=None):
    frequency = frq if frq else self.frequency
    period = 1.0 / frequency
    currentTime = time % period
    value = currentTime / period
    result = 0.0
    if value < 0.0:
      result = value * 4
    elif value > 0.8:
      result = value * 4 - 4.0
    else:
      result = 0
    wave = self.amplitude * result
    return wave

  def mixwave(self, time):
    _step = 3 + int(sin(pi * time) * 10)
    steps = _step if _step else 1
    wave01 = self.square(time) * self.tone_triangle(time, _step)
    wave02 = self.white_noise(time) * self.tone_triangle(time, 1)
    wave = wave01 + wave02
    return wave


class Synth:
  def __init__(self, parent):
    self.parent: ui.View = parent  # 親となるui.View
    self.audioEngine: AVAudioEngine
    self.sampleRate: float = 44100.0  # set_up メソッド: outputNode より確定
    self.deltaTime: float = 0.0  # 1/sampleRate 時間間隔
    self.timex: float = 0.0  # Render の間隔カウンター

    self.render_block = ObjCBlock(
      self.source_node_render,
      restype=OSStatus,
      argtypes=[
        ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p, ctypes.c_void_p,
        ctypes.POINTER(AudioBufferList)
      ])

    self.tap_block = ObjCBlock(
      self.audio_node_tap,
      restype=None,
      argtypes=[
        ctypes.c_void_p,
        ctypes.c_void_p,
        ctypes.c_void_p,
      ])

    self.set_up()

  def set_up(self):
    audioEngine = AVAudioEngine.new()
    sourceNode = AVAudioSourceNode.alloc()
    mainMixer = audioEngine.mainMixerNode()
    outputNode = audioEngine.outputNode()
    format = outputNode.inputFormatForBus_(0)

    delay = AVAudioUnitDelay.new()
    delay.delayTime = 0.3
    delay.feedback = 80

    self.sampleRate = format.sampleRate()
    self.deltaTime = 1 / self.sampleRate

    inputFormat = AVAudioFormat.alloc(
    ).initWithCommonFormat_sampleRate_channels_interleaved_(
      format.commonFormat(), self.sampleRate, CHANNEL, format.isInterleaved())

    sourceNode.initWithFormat_renderBlock_(inputFormat, self.render_block)

    audioEngine.attachNode_(sourceNode)
    sourceNode.volume = 0.1

    audioEngine.attachNode_(delay)

    audioEngine.connect_to_format_(sourceNode, delay, inputFormat)
    audioEngine.connect_to_format_(delay, mainMixer, inputFormat)

    audioEngine.connect_to_format_(mainMixer, outputNode, inputFormat)

    audioEngine.prepare()

    _bufsize = 64 * 64  # 取得する情報量
    mainMixer.installTapOnBus_bufferSize_format_block_(
      0, _bufsize, inputFormat, self.tap_block)
    self.audioEngine = audioEngine

  def source_node_render(self,
                         _cmd: ctypes.c_void_p,
                         _isSilence_ptr: ctypes.c_void_p,
                         _timestamp_ptr: ctypes.c_void_p,
                         frameCount: ctypes.c_void_p,
                         outputData_ptr: ctypes.POINTER) -> OSStatus:
    ablPointer = outputData_ptr.contents
    for frame in range(frameCount):
      sampleVal = self.parent.toneGenerator(self.timex)

      self.timex += self.deltaTime

      for bufferr in range(ablPointer.mNumberBuffers):
        _mData = ablPointer.mBuffers[bufferr].mData
        _pointer = ctypes.POINTER(ctypes.c_float * frameCount)
        buffer = ctypes.cast(_mData, _pointer).contents
        buffer[frame] = sampleVal
    return 0

  def audio_node_tap(self, _cmd, buffer, when):
    buf = ObjCInstance(buffer)
    _array = buf.floatChannelData()[0]
    # 画像生成用に配列組み替え
    np_buff = np.ctypeslib.as_array(_array, (256, 16))
    with BytesIO() as bIO:
      matplotlib.image.imsave(bIO, np_buff + 1, format='png')
      img = ui.Image.from_data(bIO.getvalue())
      self.parent.visualize_view.image = img

  def start(self):
    self.audioEngine.startAndReturnError_(None)

  def stop(self):
    self.audioEngine.stop()


class View(ui.View):
  def __init__(self, *args, **kwargs):
    ui.View.__init__(self, *args, **kwargs)

    self.visualize_view = ui.ImageView()
    self.visualize_view.bg_color = 0
    self.visualize_view.flex = 'WH'
    #self.visualize_view.flex = 'TBRL'
    self.add_subview(self.visualize_view)

    self.type_osc = ui.Slider()
    self.level_frq = ui.Slider()
    self.osc_log = ui.Label()
    self.frq_log = ui.Label()

    self.osc = Oscillator()
    self.toneGenerator: Oscillator
    self.setup_osc()

    self.synth = Synth(self)
    self.synth.start()

  def setup_osc(self):
    self.toneGenerator = self.osc.wave_types[0]
    self.setup_type_osc()
    self.setup_frq_level()

  def setup_type_osc(self):
    # --- slider
    self.type_len = len(self.osc.wave_types) - 1
    self.type_osc.continuous = False
    self.type_osc.value = 0
    self.type_osc.flex = 'W'
    self.type_osc.action = self.change_osc
    self.add_subview(self.type_osc)

    # --- label
    self.osc_log.text = self.toneGenerator.__name__
    self.osc_log.bg_color = 1
    self.osc_log.flex = 'W'
    self.osc_log.size_to_fit()
    self.add_subview(self.osc_log)

  def change_osc(self, sender):
    val = int(sender.value * self.type_len)
    self.toneGenerator = self.osc.wave_types[val]
    self.type_osc.value = val / self.type_len
    self.osc_log.text = self.toneGenerator.__name__

  def setup_frq_level(self):
    # --- slider
    self.max_level_frq = 880.0
    self.level_frq.value = self.osc.frequency / self.max_level_frq
    self.level_frq.flex = 'W'
    self.level_frq.action = self.change_frq
    self.add_subview(self.level_frq)

    # --- label
    self.frq_log.text = f'{self.osc.frequency}'
    self.frq_log.bg_color = 1
    self.frq_log.flex = 'W'
    self.frq_log.size_to_fit()
    self.add_subview(self.frq_log)

  def change_frq(self, sender):
    val = sender.value * self.max_level_frq
    self.osc.frequency = val
    self.frq_log.text = f'{self.osc.frequency}'

  def layout(self):
    # --- slider
    self.type_osc.y = self.type_osc.height
    self.level_frq.y = self.type_osc.height + self.type_osc.y * 2

    # --- label
    self.osc_log.y = self.frame.height / 2 - self.osc_log.height
    self.frq_log.y = self.osc_log.y + self.frq_log.height

    logs_width = self.frame.width / 2
    self.osc_log.width = self.frq_log.width = logs_width

  def will_close(self):
    self.synth.stop()


if __name__ == '__main__':
  view = View()
  #view.present()
  # スライダー操作時にView が動いてしまうため
  view.present(style='fullscreen', orientations=['portrait'])

参考文献

• 【iOS】Core Audioでシンセサイザーを作る - Qiita
• Building a Synthesizer in Swift. Making audio waveforms with… | by SwiftMoji | Better Programming
• Building a Signal Generator | Apple Developer Documentation
• AVAudioSourceNode, AVAudioSinkNode or How I Deleted a Repo in Less Than 24 hours

次回は

2回に渡り、AVAudioSourceNode を使った音の生成プログラミングを行いました。

Objective-C、objc_util、サウンドデータ、Python モジュールと、さまざまな要素が入り混じり、大変でしたね。

このコードをベースに足したり引いたりして、サウンドプログラミングを楽しんでいただけますと幸いです。

過去に実装した、低レイヤーの音を出すリポジトリはこちらです。

pome-ta/pysta-sine_wave001

次回は、SceneKit Framework（Pythonista3 のscene モジュールではない）に入っていきます。

楽しい3DCG の世界が待ってますよー。

ここまで、読んでいただきありがとうございました。

せんでん

Discord

Pythonista3 の日本語コミュニティーがあります。みなさん優しくて、わからないところも親身に教えてくれるのでこの機会に覗いてみてください。

書籍

iPhone/iPad でプログラミングする最強の本。

• Python や Jupyter で iPhone/iPad 先端機能を簡単･自由にプログラミング！「活用篇」：hirax

• Python や Jupyter で iPhone/iPad 先端機能を簡単･自由にプログラミング！「土台篇」：hirax

その他

• サンプルコード

Pythonista3 Advent Calendar 2022 でのコードをまとめているリポジトリがあります。

コードのエラーや変なところや改善点など。ご指摘やPR お待ちしておりますー

• Twitter

なんしかガチャガチャしていますが、お気兼ねなくお声がけくださいませー

やれるか、やれないか。ではなく、やるんだけども、紹介説明することは尽きないと思うけど、締め切り守れるか？って話よ！（クズ）

Pythonista3 Advent Calendar 2022 https://t.co/JKUxA525Pt #Qiita

— pome-ta (@pome_ta93) November 4, 2022

• GitHub

基本的にGitHub にコードをあげているので、何にハマって何を実装しているのか観測できると思います。