gocha124の日記

ごちゃごちゃ書きます

Gitと連携してdocker build 

cd ~/build_httpd
git init
echo ".git" > .dockerignore
vi .git/hooks/pre-commit
chmod u+x .git/hooks/pre-commit

~/build_httpd/.git/hooks/pre-commit

#!/bin/bash

../build_info

if [[ -z $DOCKER_REPO || -z $DOCKER_TAG ]]; then
  echo "you need to set ¥$DOCKER_REPO and ¥$DOCKER_TAG in build_info."
  exit 1
fi

if docker images $DOCKER_REPO | awk '{print $2}' | grep -q $DOCKER_TAG; then
  echo "Image $DOCKER_REPO:$DOCKER_TAG already exists."
  echo "Please specify a different tag in build_info."
  exit 1
fi

echo "Build an image $DOCKER_REPO:$DOCKER_TAG..."
docker build -t $DOCKER_REPO:$DOCKER_TAG .
rc=$?

if [[ $rc -eq 0 ]]; then
  echo "Build succeeded. Your change will be committed."
  echo ""
  exit 0
else
  echo "Build failed. Your change won't be committed."
  echo ""
  exit 1
fi

~/build_httpd/build_info

DOCKER_REPO=txcdb738/httpd
DOCKER_TAG=ver1.2

Dockerの操作2

Docker Reigstoryのイメージを取得する。

[centos@ip-172-31-39-215 ~]$ sudo docker pull registry
Using default tag: latest
Trying to pull repository docker.io/library/registry ... 
latest: Pulling from docker.io/library/registry
486039affc0a: Pull complete 
ba51a3b098e6: Pull complete 
8bb4c43d6c8e: Pull complete 
6f5f453e5f2d: Pull complete 
42bc10b72f42: Pull complete 
Digest: sha256:7d081088e4bfd632a88e3f3bcd9e007ef44a796fddfe3261407a3f9f04abe1e7
Status: Downloaded newer image for docker.io/registry:latest
[centos@ip-172-31-39-215 ~]$

Docker ResistoryのImageが取得できている。

[centos@ip-172-31-39-215 ~]$ sudo docker images
REPOSITORY                  TAG                 IMAGE ID            CREATED             SIZE
txcdb738/httpd              ver1.0              4ae50559bf03        25 hours ago        302 MB
docker.io/registry          latest              708bc6af7e5e        3 weeks ago         25.8 MB
docker.io/enakai00/centos   centos6             b8c94c5d2d7c        4 years ago         203 MB
docker.io/enakai00/centos   centos7             3bd78cf8ed76        4 years ago         172 MB
docker.io/enakai00/centos   latest              3bd78cf8ed76        4 years ago         172 MB

ローカルDocker Resitoryを起動する。

[centos@ip-172-31-39-215 ~]$ sudo docker run -d -p 5000:5000 registry
7eca1b83a6cdfb4914c90a2502c792507040ab41b84a8e5e72bd6b9a8f31d84e

[centos@ip-172-31-39-215 ~]$ sudo docker ps --format "{{.ID}}\t{{.Image}}\t{{.Ports}}"
7eca1b83a6cd	registry	0.0.0.0:5000->5000/tcp
[centos@ip-172-31-39-215 ~]$ sudo docker images

タグをはって、

[centos@ip-172-31-39-215 ~]$ sudo docker tag txcdb738/httpd:ver1.0 localhost:5000/txcdb738/httpd:ver1.0
[centos@ip-172-31-39-215 ~]$ sudo docker images txcdb738/httpd
REPOSITORY          TAG                 IMAGE ID            CREATED             SIZE
txcdb738/httpd      ver1.0              4ae50559bf03        25 hours ago        302 MB
[centos@ip-172-31-39-215 ~]$ sudo docker images localhost:5000/txcdb738/httpd
REPOSITORY                      TAG                 IMAGE ID            CREATED             SIZE
localhost:5000/txcdb738/httpd   ver1.0              4ae50559bf03        25 hours ago        302 MB

ローカルRegistoryへプッシュする。

[centos@ip-172-31-39-215 ~]$ sudo docker push localhost:5000/txcdb738/httpd
The push refers to a repository [localhost:5000/txcdb738/httpd]
69b6f066ea18: Pushed 
5f70bf18a086: Pushed 
8d12f3483b2e: Pushed 
ver1.0: digest: sha256:5653e8ad82193dd6c34efe3dfb096b9d7041e2dffad53cac0b9b295ccba1fd50 size: 1153

Dockerfileを記述する。

[centos@ip-172-31-39-215 ~]$ vi ~/build_httpd/Dockerfile 
[centos@ip-172-31-39-215 ~]$ cat ~/build_httpd/Dockerfile 

FROM enakai00/centos:centos6
MAINTAINER txcdb738

RUN yum -y install httpd
ADD index.html /var/www/html/index.htlm
CMD service httpd start && bash

Dockerfileを使ってビルドする。

[centos@ip-172-31-39-215 ~]$ sudo docker build -t txcdb738/httpd:ver1.1 ~/build_httpd

ビルドしたイメージでhttpdを起動する。

[centos@ip-172-31-39-215 ~]$ sudo docker run -itd -p 8000:80 --name web01 txcdb738/httpd:ver1.1
5d7dbf821310d649bfac1df0d9b4e722b2cbb71135c5d251bca2c0e15a285661

dockerのログ

[centos@ip-172-31-39-215 ~]$ sudo docker logs web01
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.3 for ServerName
                                                           [  OK  ]

停止して削除する。

[centos@ip-172-31-39-215 ~]$ sudo docker stop web01
web01
[centos@ip-172-31-39-215 ~]$ sudo docker rm web01
web01

Dockerの操作

基本操作めも

[centos@ip-172-31-39-215 ~]$ sudo yum -y install net-tools
[centos@ip-172-31-39-215 ~]$ sudo yum -y update
[centos@ip-172-31-39-215 ~]$ sudo reboot
[centos@ip-172-31-39-215 ~]$ sudo systemctl stop firewalld.service
Failed to stop firewalld.service: Unit firewalld.service not loaded.
[centos@ip-172-31-39-215 ~]$ sudo systemctl mask firewalld.service
Created symlink from /etc/systemd/system/firewalld.service to /dev/null.
[centos@ip-172-31-39-215 ~]$ sudo yum -y install docker
[centos@ip-172-31-39-215 ~]$ sudo systemctl enable docker.service
Created symlink from /etc/systemd/system/multi-user.target.wants/docker.service to /usr/lib/systemd/system/docker.service.
[centos@ip-172-31-39-215 ~]$ sudo systemctl start docker.service
[centos@ip-172-31-39-215 ~]$ sudo docker search enakai00
[centos@ip-172-31-39-215 ~]$ sudo docker pull -a enakai00/centos
[centos@ip-172-31-39-215 ~]$ sudo docker images
[centos@ip-172-31-39-215 ~]$ sudo docker run -it --name centos01 enakai00/centos:centos6 /bin/bash
[root@b6cf0659e158 /]# df -h /
Filesystem      Size  Used Avail Use% Mounted on
overlay         8.0G  2.0G  6.1G  24% /
[root@b6cf0659e158 /]# ps -ef
UID        PID  PPID  C STIME TTY          TIME CMD
root         1     0  0 15:05 ?        00:00:00 /bin/bash
root        16     1  0 15:07 ?        00:00:00 ps -ef
[root@b6cf0659e158 /]# ifconfig
eth0      Link encap:Ethernet  HWaddr 02:42:AC:11:00:02  
          inet addr:172.17.0.2  Bcast:0.0.0.0  Mask:255.255.0.0
          inet6 addr: fe80::42:acff:fe11:2/64 Scope:Link
          UP BROADCAST RUNNING MULTICAST  MTU:1500  Metric:1
          RX packets:16 errors:0 dropped:0 overruns:0 frame:0
          TX packets:8 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:0 
          RX bytes:1312 (1.2 KiB)  TX bytes:656 (656.0 b)

lo        Link encap:Local Loopback  
          inet addr:127.0.0.1  Mask:255.0.0.0
          inet6 addr: ::1/128 Scope:Host
          UP LOOPBACK RUNNING  MTU:65536  Metric:1
          RX packets:0 errors:0 dropped:0 overruns:0 frame:0
          TX packets:0 errors:0 dropped:0 overruns:0 carrier:0
          collisions:0 txqueuelen:1000 
          RX bytes:0 (0.0 b)  TX bytes:0 (0.0 b)

[root@b6cf0659e158 /]# route -n
Kernel IP routing table
Destination     Gateway         Genmask         Flags Metric Ref    Use Iface
0.0.0.0         172.17.0.1      0.0.0.0         UG    0      0        0 eth0
172.17.0.0      0.0.0.0         255.255.0.0     U     0      0        0 eth0
[centos@ip-172-31-39-215 ~]$ sudo docker ps
[centos@ip-172-31-39-215 ~]$ sudo docker attach centos01
[root@b6cf0659e158 /]# yum -y install httpd
[centos@ip-172-31-39-215 ~]$ sudo docker stop centos01
[centos@ip-172-31-39-215 ~]$ sudo docker ps -a
CONTAINER ID        IMAGE                     COMMAND             CREATED             STATUS                       PORTS               NAMES
b6cf0659e158        enakai00/centos:centos6   "/bin/bash"         5 minutes ago       Exited (137) 8 seconds ago                       centos01
[centos@ip-172-31-39-215 ~]$ sudo docker start centos01
centos01
[centos@ip-172-31-39-215 ~]$ sudo docker commit centos01 txcdb738/httpd:ver1.0
sha256:4ae50559bf0383c33c34800b3ef50d7a0455a9c29f445f387ae2e4a12b4bc3a3
[centos@ip-172-31-39-215 ~]$ sudo docker images txcdb738/httpd
[centos@ip-172-31-39-215 ~]$ sudo docker attach centos01
[centos@ip-172-31-39-215 ~]$ sudo docker run -it -p 8000:80 --name web01 txcdb738/httpd:ver1.0 /bin/bash
[root@03f2fdf05c94 /]# 
[root@03f2fdf05c94 /]# echo '<h1>Hello, Docker!</h1>' > /var/www/html/index.html
[root@03f2fdf05c94 /]# service httpd start
Starting httpd: httpd: Could not reliably determine the server's fully qualified domain name, using 172.17.0.2 for ServerName
                                                           [  OK  ]
[root@03f2fdf05c94 /]# 
[centos@ip-172-31-39-215 ~]$

10分間チュートリアル　深層学習モデルをトレーニング　をやってみた

やってみた。

ssh -L localhost:8888:localhost:8888 -i ./txcdbxxx.pem ubuntu@54.238.183.142

The authenticity of host '54.238.183.142 (54.238.183.142)' can't be established.
ECDSA key fingerprint is SHA256:IM/fUO2YHtfLV2ykb4wyitm7/1uyuW4/28TvnL+s40o.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added '54.238.183.142' (ECDSA) to the list of known hosts.
=============================================================================
       __|  __|_  )
       _|  (     /   <span style="color: #0000cc">Deep Learning Base AMI (Ubuntu 16.04) Version 21.0</span>
      ___|\___|___|
=============================================================================

Welcome to Ubuntu 16.04.6 LTS (GNU/Linux 4.4.0-1101-aws x86_64v)

Nvidia driver version: 418.87.01
CUDA versions available: cuda-10.0 cuda-10.1 cuda-8.0 cuda-9.0 cuda-9.2 
Default CUDA version is 10.0 
Libraries: cuDNN, NCCL, Intel MKL-DNN

AWS Deep Learning AMI Homepage: https://aws.amazon.com/machine-learning/amis/
Developer Guide and Release Notes: https://docs.aws.amazon.com/dlami/latest/devguide/what-is-dlami.html
Support: https://forums.aws.amazon.com/forum.jspa?forumID=263
For a fully managed experience, check out Amazon SageMaker at https://aws.amazon.com/sagemaker
When using INF1 type instances, please update regularly using the instructions at: https://github.com/aws/aws-neuron-sdk/tree/master/release-notes
=============================================================================

 * Documentation:  https://help.ubuntu.com
 * Management:     https://landscape.canonical.com
 * Support:        https://ubuntu.com/advantage

 * Multipass 1.0 is out! Get Ubuntu VMs on demand on your Linux, Windows or
   Mac. Supports cloud-init for fast, local, cloud devops simulation.

     https://multipass.run/

  Get cloud support with Ubuntu Advantage Cloud Guest:
    http://www.ubuntu.com/business/services/cloud

22 packages can be updated.
0 updates are security updates.


ubuntu@ip-172-31-41-190:~$ 

ubuntu@ip-172-31-41-190:~$ aws configure
AWS Access Key ID [****************6YBF]: 
AWS Secret Access Key [****************qSEG]: 
Default region name [None]: 
Default output format [None]: 
ubuntu@ip-172-31-41-190:~$ $(aws ecr get-login --region us-east-1 --no-include-email --registry-ids 763104351884)
WARNING! Using --password via the CLI is insecure. Use --password-stdin.
WARNING! Your password will be stored unencrypted in /home/ubuntu/.docker/config.json.
Configure a credential helper to remove this warning. See
https://docs.docker.com/engine/reference/commandline/login/#credentials-store

Login Succeeded
ubuntu@ip-172-31-41-190:~$ pwd
/home/ubuntu

ubuntu@ip-172-31-41-190:~$ docker run -it 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:1.13-cpu-py36-ubuntu16.04
Unable to find image '763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:1.13-cpu-py36-ubuntu16.04' locally
1.13-cpu-py36-ubuntu16.04: Pulling from tensorflow-training
35b42117c431: Pull complete 
ad9c569a8d98: Pull complete 
293b44f45162: Pull complete 
0c175077525d: Pull complete 
1365b8b5a858: Pull complete 
48bc23b4c956: Pull complete 
a0eb117d191e: Pull complete 
66b02ff5f427: Pull complete 
b379b7a5ac86: Pull complete 
25835df539e7: Pull complete 
42fc20c5c0db: Pull complete 
0971bc98571d: Pull complete 
367408011efb: Pull complete 
ee5e0f5ff66e: Pull complete 
acae1d207982: Pull complete 
Digest: sha256:ab0d1015d08ee0323d98f77bf00dec2f0b98d90a665b0f2fad5e461759cbb9cc
Status: Downloaded newer image for 763104351884.dkr.ecr.us-east-1.amazonaws.com/tensorflow-training:1.13-cpu-py36-ubuntu16.04
root@5f64fd0b0055:/# pwd
/

root@5f64fd0b0055:/# git clone https://github.com/fchollet/keras.git
Cloning into 'keras'...
remote: Enumerating objects: 32987, done.
remote: Total 32987 (delta 0), reused 0 (delta 0), pack-reused 32987
Receiving objects: 100% (32987/32987), 13.11 MiB | 6.29 MiB/s, done.
Resolving deltas: 100% (24101/24101), done.
Checking connectivity... done.
root@5f64fd0b0055:/# python keras/examples/mnist_cnn.py
Using TensorFlow backend.
Downloading data from https://s3.amazonaws.com/img-datasets/mnist.npz
11493376/11490434 [==============================] - 2s 0us/step
x_train shape: (60000, 28, 28, 1)
60000 train samples
10000 test samples
WARNING:tensorflow:From /usr/local/lib/python3.6/site-packages/tensorflow/python/framework/op_def_library.py:263: colocate_with (from tensorflow.python.framework.ops) is deprecated and will be removed in a future version.
Instructions for updating:
Colocations handled automatically by placer.
WARNING:tensorflow:From /usr/local/lib/python3.6/site-packages/keras/backend/tensorflow_backend.py:3445: calling dropout (from tensorflow.python.ops.nn_ops) with keep_prob is deprecated and will be removed in a future version.
Instructions for updating:
Please use `rate` instead of `keep_prob`. Rate should be set to `rate = 1 - keep_prob`.
WARNING:tensorflow:From /usr/local/lib/python3.6/site-packages/tensorflow/python/ops/math_ops.py:3066: to_int32 (from tensorflow.python.ops.math_ops) is deprecated and will be removed in a future version.
Instructions for updating:
Use tf.cast instead.
Train on 60000 samples, validate on 10000 samples
Epoch 1/12
2020-02-10 13:14:34.544621: I tensorflow/core/platform/cpu_feature_guard.cc:141] Your CPU supports instructions that this TensorFlow binary was not compiled to use: AVX512F
2020-02-10 13:14:34.548961: I tensorflow/core/platform/profile_utils/cpu_utils.cc:94] CPU Frequency: 3000000000 Hz
2020-02-10 13:14:34.549112: I tensorflow/compiler/xla/service/service.cc:150] XLA service 0x55de7f0 executing computations on platform Host. Devices:
2020-02-10 13:14:34.549333: I tensorflow/compiler/xla/service/service.cc:158]   StreamExecutor device (0): <undefined>, <undefined>
2020-02-10 13:14:34.549506: I tensorflow/core/common_runtime/process_util.cc:71] Creating new thread pool with default inter op setting: 2. Tune using inter_op_parallelism_threads for best performance.
60000/60000 [==============================] - 57s 948us/step - loss: 0.2741 - acc: 0.9148 - val_loss: 0.0602 - val_acc: 0.9805
Epoch 2/12
60000/60000 [==============================] - 56s 930us/step - loss: 0.0922 - acc: 0.9723 - val_loss: 0.0408 - val_acc: 0.9856
Epoch 3/12
60000/60000 [==============================] - 56s 934us/step - loss: 0.0700 - acc: 0.9793 - val_loss: 0.0336 - val_acc: 0.9890
Epoch 4/12
60000/60000 [==============================] - 56s 930us/step - loss: 0.0581 - acc: 0.9830 - val_loss: 0.0331 - val_acc: 0.9887
Epoch 5/12
60000/60000 [==============================] - 56s 930us/step - loss: 0.0516 - acc: 0.9845 - val_loss: 0.0341 - val_acc: 0.9879
Epoch 6/12
60000/60000 [==============================] - 56s 930us/step - loss: 0.0446 - acc: 0.9863 - val_loss: 0.0286 - val_acc: 0.9909
Epoch 7/12
60000/60000 [==============================] - 56s 931us/step - loss: 0.0395 - acc: 0.9880 - val_loss: 0.0281 - val_acc: 0.9904
Epoch 8/12
60000/60000 [==============================] - 56s 930us/step - loss: 0.0380 - acc: 0.9879 - val_loss: 0.0284 - val_acc: 0.9911
Epoch 9/12
60000/60000 [==============================] - 56s 931us/step - loss: 0.0354 - acc: 0.9890 - val_loss: 0.0276 - val_acc: 0.9912
Epoch 10/12
60000/60000 [==============================] - 56s 931us/step - loss: 0.0300 - acc: 0.9905 - val_loss: 0.0307 - val_acc: 0.9909
Epoch 11/12
60000/60000 [==============================] - 56s 928us/step - loss: 0.0284 - acc: 0.9910 - val_loss: 0.0277 - val_acc: 0.9902
Epoch 12/12
60000/60000 [==============================] - 56s 930us/step - loss: 0.0275 - acc: 0.9913 - val_loss: 0.0280 - val_acc: 0.9908
Test loss: 0.0280409380890409
Test accuracy: 0.9908
root@5f64fd0b0055:/#

10分間チュートリアル PostgreSQL データベースを作成して接続する方法 – AWSやってみた

10分間チュートリアル PostgreSQL データベースを作成して接続する方法 – AWSをやってみた。
マネージメントコンソールでPostgreSQLデータベースは簡単に作成できた。
しかしSQL Workbenchから接続エラーでつまずく。
パッとみたところ設定間違えが２箇所。
パブリックアクセシビリティがありになっていない。
セキュリティグループのインバウンドの設定が許可になっていない。
対応できて接続成功。
データベースを削除して終了。

10分間チュートリアル　Windows 仮想マシンの起動方法 – AWSをやってみた

EC2でのWindows AMIの作成とアクセスのチュートリアル。
EC2からのWindowsAMIの作成は、マネージメントコンソールからふつうに作成するだけ。
アクセスはEC2で作成したキーペアを使う。キーペアから作成される、おそらく公開キーと秘密キーからWindowsのAdministratorのパスワードが生成されている。これをリモートデスクトップ、RDPでURL、ユーザ、パスワードに指定することでアクセスできる。
さすがよく考えてあるな。