Configuring TCP service on Elastic Beanstalk

It's possible, but you need to recompile nginx to add tcp proxy module. Fret not, I got you covered.

.ebextensions/00-rebuild-nginx.config

packages:
  yum:
    git: []
    gcc: []
    gcc-c++: []
    make: []
    zlib-devel: []
    pcre-devel: []
    openssl-devel: []
    libxml2-devel: []
    libxslt-devel: []
    patch: []
    gd-devel: []
    perl-ExtUtils-Embed: []
    GeoIP-devel: []
    gperftools-devel: []
files:
  "/root/build-nginx.sh" :
    mode: "000755"
    owner: root
    group: root
    content: |
      rm -rf build
      mkdir ./build ; cd ./build
      wget -O - http://nginx.org/download/nginx-1.6.2.tar.gz | tar xfvz -
      git clone git://github.com/yaoweibin/nginx_tcp_proxy_module

      cd nginx-1.6.2/
      patch -p1 < ../nginx_tcp_proxy_module/tcp.patch

      ./configure --add-module=../nginx_tcp_proxy_module \
      --prefix=/usr/share/nginx --sbin-path=/usr/sbin/nginx \
      --conf-path=/etc/nginx/nginx.conf --error-log-path=/var/log/nginx/error.log \
      --http-log-path=/var/log/nginx/access.log \
      --http-client-body-temp-path=/var/lib/nginx/tmp/client_body \
      --http-proxy-temp-path=/var/lib/nginx/tmp/proxy \
      --http-fastcgi-temp-path=/var/lib/nginx/tmp/fastcgi \
      --http-uwsgi-temp-path=/var/lib/nginx/tmp/uwsgi \
      --http-scgi-temp-path=/var/lib/nginx/tmp/scgi --pid-path=/var/run/nginx.pid \
      --lock-path=/var/lock/subsys/nginx --user=nginx --group=nginx \
      --with-file-aio --with-ipv6 --with-http_ssl_module --with-http_spdy_module \
      --with-http_realip_module --with-http_addition_module --with-http_xslt_module \
      --with-http_image_filter_module --with-http_geoip_module \
      --with-http_sub_module --with-http_dav_module --with-http_flv_module \
      --with-http_mp4_module --with-http_gunzip_module --with-http_gzip_static_module \
      --with-http_random_index_module --with-http_secure_link_module \
      --with-http_degradation_module --with-http_stub_status_module --with-http_perl_module \
      --with-mail --with-mail_ssl_module --with-pcre --with-google_perftools_module \
      --with-debug \
      --with-cc-opt='-O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic' \
      --with-ld-opt=' -Wl,-E'

      make
      make install
  "/etc/nginx/nginx-tcp-proxy.conf":
    mode: "0644"
    owner: root
    group: root
    content: |
      # placeholder
      # /opt/elasticbeanstalk/hooks/appdeploy/enact/00before-flip.sh should update this

  "/opt/elasticbeanstalk/hooks/appdeploy/enact/00before-flip.sh":
    mode: "0755"
    owner: root
    group: root
    content: |
      . /opt/elasticbeanstalk/hooks/common.sh

      EB_CONFIG_DOCKER_PORT_FILE=$(/opt/elasticbeanstalk/bin/get-config container -k port_file)

      EB_CONFIG_DOCKER_STAGING_APP_FILE=$(/opt/elasticbeanstalk/bin/get-config container -k app_staging_file)
      EB_CONFIG_DOCKER_CURRENT_APP_FILE=$(/opt/elasticbeanstalk/bin/get-config container -k app_deploy_file)

      EB_CONFIG_DOCKER_IMAGE_STAGING=$(/opt/elasticbeanstalk/bin/get-config container -k staging_image)
      EB_CONFIG_DOCKER_IMAGE_CURRENT=$(/opt/elasticbeanstalk/bin/get-config container -k deploy_image)

      EB_CONFIG_HTTP_PORT=$(/opt/elasticbeanstalk/bin/get-config container -k instance_port)

      # now the STAGING container is built and running, flip nginx to the new container
      EB_CONFIG_NGINX_UPSTREAM_IP=$(docker inspect `cat $EB_CONFIG_DOCKER_STAGING_APP_FILE` | jq -r .[0].NetworkSettings.IPAddress)
      EB_CONFIG_NGINX_UPSTREAM_PORT=`cat $EB_CONFIG_DOCKER_PORT_FILE`

      # set up nginx
      cat > /etc/nginx/nginx-tcp-proxy.conf <<EOF
      tcp {
        upstream docker {
                server $EB_CONFIG_NGINX_UPSTREAM_IP:$EB_CONFIG_NGINX_UPSTREAM_PORT;
        }

        server {
              listen $EB_CONFIG_HTTP_PORT;
              proxy_pass docker;
        }

        access_log /var/log/nginx/tcp_access.log;
      }
      EOF


commands:
  00-pwd:
    command: pwd
  install-tcp_proxy_module:
    test: test -e $(2>&1 nginx -V | tr -- - '\n' | grep _module | grep tcp)
    command: chmod +x ~/build-nginx.sh && ~/build-nginx.sh

container_commands:
  00-disable-default-http:
    command: test ! -f /etc/nginx/sites-enabled/elasticbeanstalk-nginx-docker-proxy.conf || rm -f /etc/nginx/sites-enabled/elasticbeanstalk-nginx-docker-proxy.conf
  01-include-tcp-proxy-setting:
    command: grep 'nginx-tcp-proxy' /etc/nginx/nginx.conf || echo 'include       /etc/nginx/nginx-tcp-proxy.conf;' >> /etc/nginx/nginx.conf

This is way internet is important. Above text is build by gathering pieces from all over the place.

In essence it requires you to understand how EB build instance and deploy stuff.

Above configuration flags was taken out from Amazon Linux Docker AMI instance. Nothing was stripped out.

I was re-using 00flip.sh content to construct the tcp proxy config. I put the script to run before 00flip.sh so nginx will properly take it into account. Conf file was put outside nginx's conf.d and sites-available since all files there will be included under http section.

Default site also have to be disabled so we can use the port for our TCP service.

Several notes on setting up NAT (event if you are using VPC wizard)

• Always create a new security group for private instances and nat instances. This will make security group management a breeze
• Need at least 2 subnets on each availablity zone! (I don't have much network knowledge on my "Ops" profile - thus my ELB and instances are on different availability zone. Stupid, I know.)
• Make sure NAT is configured to accept inbound from private vpc

Other take: Rebuilding docker image on each deployment or during autoscale event is not scalable

Finally. T__T

万歳！！