AWSサービス一覧をスクレイピングで雑に取得する

AWSの クラウド製品 のページから、
/documents/xxx/ にリンクされているサービスを取得して一覧にする。

今回は、以前に使ったことがある、PHP Simple HTML DOM Parserを使用しました。

<?php

require_once 'simple_html_dom.php';

$url = "http://aws.amazon.com/jp/products/";

$html = file_get_html($url);

$services = [];
foreach ($html->find('div.aws-link') as $element) {
    foreach ($element->find('a') as $e) {
        $url  = $e->href;
        $text =  $e->innertext;
        if (preg_match("|/documentation/([a-zA-Z0-9\-\_]+)/|", $url, $match)) {
            $service = $match[1];
            $services[$service] = $text;
        }
    }
}

$i = 1;
foreach ($services as $key => $name) {
    printf("[%02d] https://aws.amazon.com/jp/%s/ : %s \n", $i++, $key, $name);
}

出力結果

[01] https://aws.amazon.com/jp/ec2/ : Amazon EC2
[02] https://aws.amazon.com/jp/ecs/ : Amazon ECS
[03] https://aws.amazon.com/jp/elastic-beanstalk/ : AWS Elastic Beanstalk
[04] https://aws.amazon.com/jp/lambda/ : AWS Lambda
[05] https://aws.amazon.com/jp/autoscaling/ : Auto Scaling
[06] https://aws.amazon.com/jp/elasticloadbalancing/ : Elastic Load Balancing
[07] https://aws.amazon.com/jp/vpc/ : Amazon VPC
[08] https://aws.amazon.com/jp/direct-connect/ : AWS Direct Connect
[09] https://aws.amazon.com/jp/route53/ : Amazon Route 53
[10] https://aws.amazon.com/jp/workspaces/ : Amazon WorkSpaces
[11] https://aws.amazon.com/jp/wam/ : Amazon WAM
[12] https://aws.amazon.com/jp/workdocs/ : Amazon WorkDocs
[13] https://aws.amazon.com/jp/workmail/ : Amazon WorkMail (プレビュー)
[14] https://aws.amazon.com/jp/elasticmapreduce/ : Amazon EMR
[15] https://aws.amazon.com/jp/datapipeline/ : AWS Data Pipeline
[16] https://aws.amazon.com/jp/kinesis/ : Amazon Kinesis
[17] https://aws.amazon.com/jp/machine-learning/ : Amazon Machine Learning
[18] https://aws.amazon.com/jp/redshift/ : Amazon Redshift
[19] https://aws.amazon.com/jp/codecommit/ : AWS CodeCommit
[20] https://aws.amazon.com/jp/codedeploy/ : AWS CodeDeploy
[21] https://aws.amazon.com/jp/codepipeline/ : AWS CodePipeline
[22] https://aws.amazon.com/jp/cli/ : AWS コマンドラインインターフェイス
[23] https://aws.amazon.com/jp/cloudwatch/ : Amazon CloudWatch
[24] https://aws.amazon.com/jp/cloudfront/ : Amazon CloudFront
[25] https://aws.amazon.com/jp/cloudtrail/ : AWS CloudTrail
[26] https://aws.amazon.com/jp/config/ : AWS Config
[27] https://aws.amazon.com/jp/opsworks/ : AWS OpsWorks
[28] https://aws.amazon.com/jp/servicecatalog/ : AWS Service Catalog
[29] https://aws.amazon.com/jp/aws-support/ : Trusted Advisor
[30] https://aws.amazon.com/jp/iam/ : AWS Identity and Access Management (IAM)
[31] https://aws.amazon.com/jp/cloudhsm/ : AWS CloudHSM
[32] https://aws.amazon.com/jp/directory-service/ : AWS Directory Service
[33] https://aws.amazon.com/jp/kms/ : AWS KMS
[34] https://aws.amazon.com/jp/mobile-hub/ : AWS Mobile Hub
[35] https://aws.amazon.com/jp/apigateway/ : Amazon API Gateway
[36] https://aws.amazon.com/jp/devicefarm/ : AWS Device Farm
[37] https://aws.amazon.com/jp/mobileanalytics/ : Amazon Mobile Analytics
[38] https://aws.amazon.com/jp/sdk-for-android/ : AWS Mobile SDK for Android
[39] https://aws.amazon.com/jp/sdk-for-ios/ : AWS Mobile SDK for iOS
[40] https://aws.amazon.com/jp/sdk-for-unity/ : AWS Mobile SDK for Unity
[41] https://aws.amazon.com/jp/sns/ : Amazon SNS
[42] https://aws.amazon.com/jp/s3/ : Amazon S3
[43] https://aws.amazon.com/jp/efs/ : Amazon EFS (プレビュー)
[44] https://aws.amazon.com/jp/glacier/ : Amazon Glacier
[45] https://aws.amazon.com/jp/importexport/ : AWS Import/Export
[46] https://aws.amazon.com/jp/storage-gateway/ : AWS Storage Gateway
[47] https://aws.amazon.com/jp/rds/ : Amazon RDS
[48] https://aws.amazon.com/jp/dynamodb/ : Amazon DynamoDB
[49] https://aws.amazon.com/jp/elasticache/ : Amazon ElastiCache
[50] https://aws.amazon.com/jp/appstream/ : Amazon AppStream
[51] https://aws.amazon.com/jp/cloudsearch/ : Amazon CloudSearch
[52] https://aws.amazon.com/jp/elastic-transcoder/ : Amazon Elastic Transcoder
[53] https://aws.amazon.com/jp/ses/ : Amazon SES
[54] https://aws.amazon.com/jp/sqs/ : Amazon SQS
[55] https://aws.amazon.com/jp/swf/ : Amazon SWF
[56] https://aws.amazon.com/jp/iot/ : AWS IoT
[57] https://aws.amazon.com/jp/simpledb/ : Amazon SimpleDB
[58] https://aws.amazon.com/jp/cloudformation/ : AWS CloudFormation
[59] https://aws.amazon.com/jp/inspector/ : Amazon Inspector
[60] https://aws.amazon.com/jp/waf/ : AWS WAF
[61] https://aws.amazon.com/jp/elasticsearch-service/ : Amazon Elasticsearch Service

すげぇ多いな…

追記:

適当にやったので、リンク切れになってたり、サービスじゃないやつ(cliとか)も含まれてますね …