Dec 25, 2018

Try HDP 3.0.1

Started using HDP 3.0.1. Past version I was used is 2.6.5, therefore I found many changes, new Ambari design, hive command starts beeline and so one.


One major change to me, hive view has been deleted since HDP 3.0.
https://community.hortonworks.com/questions/202444/where-is-hive-view-on-hdp-3.html

Tried Data Analytics Studio instead of that.

You can move the page by
Ambari → Services → Data Analytics Studio  → Data Analytics Studio UI
http://sandbox-hdp.hortonworks.com:30800

I guess it's BI tool from the name, but it seems the DBA tool. It has 4 components.

1. Queries
You can see query history. And Recommendations tab gives us suggested changes.
Queries
Recommendations

Query Details

Visual Explain

Configs

Timeline

DAG Info


2. Compose
This is almost the same as class hive view. We can use auto suggest.
Compose

3. Database
Partitions tab shows column name, column type and a comment if the table is partitioned. I want to see partition's numbers, size and so on.
Columns

Partitions

Storage Information

Detailed Information

Statistics

Data Preview

4. Reports
This page visualizes lineage from the query history. It looks really useful. 
Reports

Read and Write Report
Join Report

Dec 23, 2018

Implement Presto Flex Connector

When I played Apache Drill, I felt it's useful to analyze local csv and parquet. Then, I developed presto connector to support accessing for local files. This is GitHub repository.

As you may already know, Presto specify a table name like `catalog.schema.table`. The connector identify the file type from schema. Current supported types are  csv, tsv, txt, raw and excel. All types except for raw returns multi rows if the file contains EOL. A raw type returns 1 column and 1 record. I assume the raw type can be used for converting to JSON file.

`table` name will be like below. You can access local and remote file.
"file:///tmp/numbers.csv"
"https://raw.githubusercontent.com/ebyhr/presto-flex/master/src/test/resources/example-data/numbers.tsv"

Here is the entire query example. The `csv` on schema and the extension of `file:///tmp/numbers.csv` are not irrelevant. Therefore, if you change the schema to `tsv`, it returns the result split by tab.

select
  * 
from 
 flex.csv."file:///tmp/numbers.csv"
;

Current Presto doesn't have importer like PostgreSQL's COPY. This connector may useful in such casešŸ­

Dec 8, 2018

AES CBC ISO7816 and SHA256 on Java

This is Java snippet using AES/CBC/ISO7816-4Padding and SHA256. javafx.cypto doesn't support ISO7816 padding, therefore I use org.bouncycastle.

The group id and artifact is is below and my sample code uses 1.52.
groupId: org.bouncycastle
artifactId: bcprov-jdk15on
version: 1.52

import java.security.Security;

import javax.crypto.Cipher;
import javax.crypto.spec.IvParameterSpec;
import javax.crypto.spec.SecretKeySpec;

import org.apache.commons.codec.digest.DigestUtils;
import org.bouncycastle.jce.provider.BouncyCastleProvider;

public final class Encryptor {

private static final byte[] KEY = "0123456789abcdef".getBytes();
private static final byte[] INIT_VECTOR = "abcdef0123456789".getBytes();

static {
Security.insertProviderAt(new BouncyCastleProvider(), 1);
}

private static String encrypt(String value)
throws Exception
{
SecretKeySpec keySpec = new SecretKeySpec(KEY, "AES");
Cipher cipher = Cipher.getInstance("AES/CBC/ISO7816-4Padding");
cipher.init(Cipher.ENCRYPT_MODE, keySpec, new IvParameterSpec(INIT_VECTOR));
byte[] encrypted = cipher.doFinal(value.getBytes());
return DigestUtils.sha256Hex(encrypted);
}
}

Dec 7, 2018

Try HBase Java Client

This is HBase Java test example. My repository is here (https://github.com/ebyhr/hbase-embed). I've changed log level to INFO because default log level is little noisy.

import org.apache.hadoop.hbase.HBaseTestingUtility;
import org.apache.hadoop.hbase.TableNotDisabledException;
import org.apache.hadoop.hbase.client.Get;
import org.apache.hadoop.hbase.client.HBaseAdmin;
import org.apache.hadoop.hbase.client.HTable;
import org.apache.hadoop.hbase.client.Put;
import org.apache.hadoop.hbase.client.Result;
import org.apache.hadoop.hbase.util.Bytes;
import org.junit.jupiter.api.AfterAll;
import org.junit.jupiter.api.BeforeAll;
import org.junit.jupiter.api.Test;

import java.io.IOException;
import java.util.Arrays;
import java.util.List;

import static org.junit.jupiter.api.Assertions.assertEquals;
import static org.junit.jupiter.api.Assertions.assertThrows;

class HBaseTest {
    private static final HBaseTestingUtility HBASE = new HBaseTestingUtility();
    private static final byte[] TABLE_NAME = Bytes.toBytes("EMPLOYEE");
    private static final byte[] CF = Bytes.toBytes("cf1");
    private static final List PEOPLES = Arrays.asList(
            new People("å±±ē”°", 20),
            new People("ģ˜ģˆ˜", 33),
            new People("John", 18)
    );

    @BeforeAll
    static void setUp()
            throws Exception
    {
        HBASE.startMiniZKCluster();
        HBASE.startMiniCluster();

        HTable table = HBASE.createTable(TABLE_NAME, CF);
        for (People people : PEOPLES) {
            int i = PEOPLES.indexOf(people);
            String rowKey = "rowkey" + i;
            Put put = new Put(Bytes.toBytes(rowKey));
            put.add(CF, Bytes.toBytes("name"), Bytes.toBytes(people.name));
            put.add(CF, Bytes.toBytes("age"), Bytes.toBytes(people.age));
            table.put(put);
        }
    }

    @AfterAll
    static void tearDown()
            throws Exception
    {
        HBASE.cleanupTestDir();
    }

    @Test
    void testAccessTable()
            throws IOException
    {
        HTable table = new HTable(HBASE.getConfiguration(), TABLE_NAME);
        String rowKey = "rowkey1";
        Result getResult = table.get(new Get(Bytes.toBytes(rowKey)));
        assertEquals("ģ˜ģˆ˜", Bytes.toString(getResult.getValue(CF, Bytes.toBytes("name"))));
        assertEquals(33, Bytes.toInt(getResult.getValue(CF, Bytes.toBytes("age"))));
    }

    @Test
    void testDrop()
            throws Exception
    {
        HTable table = HBASE.createTable(Bytes.toBytes("TMP-TABLE1"), CF);
        HBaseAdmin admin = HBASE.getHBaseAdmin();
        assertThrows(TableNotDisabledException.class, () -> admin.deleteTable(table.getTableName()));
        admin.disableTable(table.getTableName());
        admin.deleteTable(table.getTableName());
    }

    private static class People {
        private String name;
        private int age;

        private People(String name, int age)
        {
            this.name = name;
            this.age = age;
        }
    }
}

Nov 29, 2018

Kanazawa Trip

Went to Kanazawa at Oct 27 and 28. I would recommend to eat 'Fu' in addition to the seafoods. It was really delicious.
Day 1st  é‡‘ę²¢é§…→ć²ćŒć—čŒ¶å±‹č”—→é¦™ęž—åŠ→兼六園→čæ‘ę±Ÿē”ŗåø‚å “→銭ę¹Æ→ćƒ›ćƒ†ćƒ«
Day 2nd čæ‘ę±Ÿē”ŗåø‚å “→é‡‘ę²¢21äø–ē“€åšē‰©é¤Ø→鈓ęœØå¤§ę‹™é¤Ø→é‡‘ę²¢é§…
I wanted to go Kanazawa Umimirai Library, but it was closed. I should have investigated it before booking the hotel.



Kanazawa st.

Misodare Dengaku Fu


Noren

Kenrokuen








GNOME

Sento

Omicho-Ichiba

21st Century Museum of Contemporary Art, Kanazawa








D.T. Suzuki Museum


Nov 26, 2018

LINE Developer Day 2018

Went to LINE Developer Day 2018 at Happo-en. I mainly attended the session about Data Engineering. Whereas all talks was helpful, infrastructure by Saegusa-san and Kafka by Kawamura-san were especially fascinating.
  • Opening Session - "Next LINE" LINEćŒå‰µć‚‹ę–°ćŸćŖäø–ē•Œ -
  • LINE恌ē›®ęŒ‡ć™ē†ęƒ³ć®åŗƒå‘Šćƒ—ćƒ©ćƒƒćƒˆćƒ•ć‚©ćƒ¼ćƒ 
  • LINEć®ć‚¤ćƒ³ćƒ•ćƒ©ćƒ—ćƒ©ćƒƒćƒˆćƒ•ć‚©ćƒ¼ćƒ ćÆć©ć®ć‚ˆć†ć«å¤§č¦ęØ”ć‚µćƒ¼ćƒ“ć‚¹ć‚’ć‚¹ć‚±ćƒ¼ćƒ«ć•ć›é‹ē”Øć‚³ć‚¹ćƒˆć®å°ć•ćŖć‚¤ćƒ³ćƒ•ćƒ©ć‚’ęä¾›ć—ć¦ć„ć‚‹ć®ć‹
  • Multi-Tenancy Kafka cluster for LINE services with 250 billion daily messages
  • Efficient And Invincible Big Data Platform In LINE
  • OASIS - Data Analysis Platform for Multi-tenant Hadoop Cluster
  • Closing Session LINEćŒå‰µć‚‹ē†ęƒ³ć®Developer Relations


Oct 5, 2018

Aster debug the SQLMR

Aster Analytics恮SQL-MRé–¢ę•°ć‚’ćƒ‡ćƒćƒƒć‚°ć™ć‚‹ę–¹ę³•ć§ć™。ćƒ­ć‚°ć ć‘ć§åŽŸå› ćŒē‰¹å®šć§ććš、ꛓ恫ćÆć‚µćƒćƒ¼ćƒˆćƒćƒ¼ćƒ ć§å†ē¾ć§ććŖ恄Ꙃ恫ćÆ非åøøć«å½¹ć«ē«‹ć¤Tips恋ćØę€ć„ć¾ć™。
ęµć‚ŒćØ恗恦ćÆåÆ¾č±”ć®é–¢ę•°(jar, zip)ć‚’ć‚µćƒ¼ćƒćƒ¼ć‹ć‚‰ćƒ€ć‚¦ćƒ³ćƒ­ćƒ¼ćƒ‰ć—、恝悌悒čŖ­ćæč¾¼ć‚“ć SQL-MR UDFć‚’ä½œęˆå¾Œć«ć‚¤ćƒ³ć‚¹ćƒˆćƒ¼ćƒ«、ęœ€å¾Œć«å®Ÿč”Œć§ć™。

今回Random Forestć®å®Ÿč”Œäø­ć«å…ƒćƒ‡ćƒ¼ć‚æ恌ē“„50MB恗恋ē„”ć„ć«ć‚‚é–¢ć‚ć‚‰ćšJVM恮MaxHeapSize悒120GBć¾ć§čØ­å®šć—ć¦ć‚‚OOM恫ćŖ悋ćØ恄恆äŗ‹č±”恌ē™ŗē”Ÿć—ćŸć®ć§、ćć®ę™‚ć®ćƒ‡ćƒćƒƒć‚°ć®ęµć‚Œć‚’čØ˜č¼‰ć—ć¦ćæć¾ć™。
Asterć®ć‚¤ćƒ³ć‚æćƒ•ć‚§ćƒ¼ć‚¹ć‹ć‚‰å®Ÿč”Œć—ćŸć®ćÆforest_driveé–¢ę•°ć§ć™ćŒ、AMC恧惭悰悒ē¢ŗčŖć—ćŸćØ恓悍ć‚Øćƒ©ćƒ¼ćØćŖć£ć¦ć„ćŸć®ćÆ内éƒØć§å®Ÿč”Œć•ć‚Œć¦ć„ć‚‹forest_builderć ć£ćŸć®ć§、ćć®é–¢ę•°ć‚’čŖæęŸ»ć®ć¤ć„ć§ć«äæ®ę­£ć—恦ćæć¾ć—ćŸ。

1. é–¢ę•°ć®ćƒ€ć‚¦ćƒ³ćƒ­ćƒ¼ćƒ‰
ć©ć“ć«ę ¼ē“ć•ć‚Œć¦ć„悋恋ćÆē’°å¢ƒć«ä¾å­˜ć—ć¦ć—ć¾ć„ć¾ć™ćŒ、é–¢ę•°å.zip悂恗恏ćÆé–¢ę•°å.jarć§ćƒ•ć‚”ć‚¤ćƒ«ć‚’ęŽ¢ć›ć°č¦‹ć¤ć‹ć‚‹ćÆ恚恧恙。VMē’°å¢ƒć§ćÆ仄äø‹ć®ćƒ‘ć‚¹ć«å­˜åœØć—ć¦ć„ć¾ć—ćŸ。
/data/ncluster/tmp/cleanonreboot/WorkerDaemon/tasks/218.0/tmp/filedump/public

2. ćƒ©ć‚¤ćƒ–ćƒ©ćƒŖć‚’ć‚¤ćƒ³ćƒćƒ¼ćƒˆć—ćŸćƒ—ćƒ­ć‚ø悧ć‚Æćƒˆć‚’ä½œęˆ
Javać®ćƒ—ćƒ­ć‚ø悧ć‚Æćƒˆć‚’ä½œęˆć—ć¾ć™。é–“é•ćˆć¦å…ƒć®é–¢ę•°ć‚’äøŠę›ø恍恗ćŖć„ć‚ˆć†åˆ„åć«ć—ć¦ćŠćć¾ć—ć‚‡ć†。今回ćÆforest_builder_debugć«ć—ć¾ć—ćŸ。
ćƒ©ć‚¤ćƒ–ćƒ©ćƒŖ恮čæ½åŠ ćÆIntelliJ恧恂悌恰、各jarćƒ•ć‚”ć‚¤ćƒ«ć‚’å³ć‚ÆćƒŖ惃ć‚Æ→Add as a libraryć—ć¦ć„ćć¾ć™。
ꬔ恫čæ½åŠ ć—ćŸé–¢ę•°ć®packagećØåŒć˜éšŽå±¤ć§ćƒ‡ćƒćƒƒć‚°ē”Øć®ć‚³ćƒ¼ćƒ‰ć‚‚åˆć‚ć›、jarćƒ•ć‚”ć‚¤ćƒ«ć‹ć‚‰ć‚³ćƒ”ćƒ¼&ćƒšćƒ¼ć‚¹ćƒˆć—ć¾ć™。

packagećÆforest_builderć§ć‚ć‚Œć°ä»„äø‹ć®ć‚ˆć†ć«ćŖć£ć¦ć„ć¾ć—ćŸ。
package com.asterdata.sqlmr.analytics.predictive_modeling.decision_tree.forest

3. ć‚Øćƒ©ćƒ¼ćƒ”ćƒƒć‚»ćƒ¼ć‚øć‹ć‚‰åŽŸå› ć®å “ę‰€ć‚’ęŽ¢ć™ćØscanRowsćƒ”ć‚½ćƒƒćƒ‰å†…ć®ä»„äø‹ć®éƒØåˆ†ć«ćŸć©ć‚Šē€ćć¾ć™。forćƒ«ćƒ¼ćƒ—ć®å¾Œć‚ć«objectOutputStream.reset();悒čæ½åŠ ć—ćŸćØ恓悍、å¤§å¹…ć«ćƒ”ćƒ¢ćƒŖä½æē”Ø量恌ꊑ恈悉悌、50MBć®ćƒ‡ćƒ¼ć‚æć‚‚å•é”ŒćŖćå®Ÿč”Œć§ćć‚‹ć‚ˆć†ć«ćŖć‚Šć¾ć—ćŸ。

try {
    int e = poissonGenr.nextInt();
    ObjectOutputStream objectOutputStream = (ObjectOutputStream)ex.get(var13);

    for(int j = 0; j < e; ++j) {
        objectOutputStream.writeObject(var12);
    }
    objectOutputStream.reset(); // Added
} catch (Error var10) {
    throw new IllegalUsageException("Please increase JVM size to avoid Java heap out of space");
}

4. ć‚¤ćƒ³ć‚¹ćƒˆćƒ¼ćƒ«
actć§ćƒ­ć‚°ć‚¤ćƒ³ć—、ć‚¤ćƒ³ć‚¹ćƒˆćƒ¼ćƒ«ć‚³ćƒžćƒ³ćƒ‰ć‚’å®Ÿč”Œć—ć¾ć™。ćƒ•ć‚”ć‚¤ćƒ«åć—ć‹ęŒ‡å®šć§ććŖć„ć®ć§ćƒ•ć‚”ć‚¤ćƒ«ćŒć‚ć‚‹ćƒ‡ć‚£ćƒ¬ć‚Æ惈ćƒŖć¾ć§ē§»å‹•ć—ć¦ćŠćåæ…č¦ćŒć‚ć‚Šć¾ć™。
\install forest_builder_debug.jar
(ć‚¢ćƒ³ć‚¤ćƒ³ć‚¹ćƒˆćƒ¼ćƒ«ć®å “åˆćÆ\remove forest_builder_debug.jar恧恙)

5. å®Ÿč”Œ
å®Ÿč”Œę™‚ćÆć‚Øćƒ©ćƒ¼ćØćŖć£ćŸéš›ć«ä½æē”Ø恗恦恄悋ć‚Æć‚ØćƒŖć®é–¢ę•°åéƒØåˆ†ć‚’å…„ć‚Œę›æ恈悋恠恑恧恙。

Oct 4, 2018

Bulk Insert to Teradata using Python ​


This snipet is bulk-loading csv to Teradata via Python. Recently teradatasql was released, but this code uses PyTd. If you haven't setup PyTd, please install the library by `pip install teradata`.

import teradata
import csv

udaExec = teradata.UdaExec()
session = udaExec.connect("tdpid")
data = list(csv.reader(open("testExecuteManyBach.csv")))
batchsize = 10000
for num in range(0, len(data), batchsize):
    session.executemany("insert into testExecuteManyBatch values (?, ?, ?, ?)"),
    data[num:num+batchsize], batch=True)


The points are batch=True and specifying batchsize. If you don't set the batchsize or the size is too large, it will be failed (forgot the actual message though). The performance in my environment (1 node) was 10,000 rows/sec. The table has 4 columns. I assume tens of thousands looks fine, but more rows should be imported with FastLoad or MLOAD.

Aug 25, 2018

XMLAGG in Teradata

SQLServer悄PostgreSQL恫恂悋string_agg悄MySQL恮group_concatć§ć™ćŒę®‹åæµćŖ恓ćØ恫Teradata恧ćÆ存åœØć—ć¾ć›ć‚“。ä»£ć‚ć‚Šć«ćØ恄恆čØ³ć§ćÆćŖ恄恧恙恌、xmlaggćØ恄恆集ē“„é–¢ę•°ćŒć‚ć‚Š、恓悌悒ä½æ恆ćØåŒć˜ć‚ˆć†ćŖ恓ćØćŒå®Ÿē¾ć§ćć¾ć™。

ć¾ćšćÆćƒ‡ćƒ¼ć‚æ悒ęŗ–å‚™ć—ć¾ć™。
drop table test_xml_agg
;
create table test_xml_agg (
 c1 int
,c2 int
,c3 varchar(10)
)
;
insert into test_xml_agg values (1,1,'hello');
insert into test_xml_agg values (1,2,'world');
insert into test_xml_agg values (2,1,'this');
insert into test_xml_agg values (2,2,'is');
insert into test_xml_agg values (2,3,'xmlagg');

äøŠčØ˜ć®ćƒ‡ćƒ¼ć‚æ悒1列ē›®ć§é›†ē“„恗、2列ē›®ć®é †ē•Ŗ恧äø€åŗ¦3列ē›®ć‚’ęØŖć«å±•é–‹ć—ć‚«ćƒ³ćƒžć§ēµåˆć™ć‚‹ćØ恄恆ć‚Æć‚ØćƒŖ悒ę›ø恄恦ćæć¾ć™。
select
 c1  
 ,trim(trailing ',' from xmlagg(c3 || ',' order by c2) (varchar(100))) as string_agg
from test_xml_agg
group by 1
;

Result Set
c1 string_agg
1  hello, world
2  this, is, xmlagg


string_agg恫ęÆ”ć¹ć‚‹ćØć”ć”ć‚ƒć”ć‚ƒć—ć¦ć„ć¾ć™ćŒ、ć¾ćšxmlagg(c3 || ',' order by c2)恧2列ē›®ć®ę˜‡é †ć§3列ē›®ć‚’ēµåˆć—ć¦ć„ćć“ćØ悒č”Øć—ć¦ć„ć¾ć™。ē¶šć„恦(varchar(100))ć§åž‹ć‚’sysudtlib.xml恋悉varcharć«ć‚­ćƒ£ć‚¹ćƒˆć—、ęœ€å¾Œć«ęœ«å°¾ć®ć‚«ćƒ³ćƒžć‚’é™¤ć„ć¦ć„ć¾ć™。

2018/10/19čæ½čؘ
tdstats.udfconcatćØć„ć†é–¢ę•°ćŒć²ćć‹ć«å­˜åœØ恙悋ćØåŒåƒšćŒę•™ćˆć¦ćć‚Œć¾ć—ćŸ。AutoStats恧ēµ±č؈åÆ¾č±”ć‚«ćƒ©ćƒ åć‚’é€£ēµć™ć‚‹ćŸć‚ć®é–¢ę•°ćØ恮恓ćØ恧恙。
UDFCONCAT

Aug 18, 2018

GCPUG BD & ML 2018

Google Cloud Next 2018 Extended BigData & ML Dayć«č”Œć£ć¦ćć¾ć—ćŸ。前半ćÆćƒ”ćƒ¢ćƒ¬ćƒ™ćƒ«ć§ć™ćŒä»Šå›žć®ćƒŸćƒ¼ćƒˆć‚¢ćƒƒćƒ—ć«ć¤ć„ć¦、後半ćÆBQMLćØAster恮ē°”単ćŖęÆ”č¼ƒć«ć¤ć„ć¦ę›ø恄恦ćæć¾ć™。BigQuery恧Hardware Acceleration悒ä½æć£ć¦ć‚‹ć®ćÆåˆć‚ć¦ēŸ„ć£ćŸć‚“ć§ć™ćŒ、ēµę§‹é©šćć§ć—ćŸ。

# BigQuery New Feature by satoluxx
BQML
- 今恮ćØ恓悍ē·šå½¢å›žåø°ćØ惭ć‚øć‚¹ćƒ†ć‚£ćƒƒć‚Æ回åø°ć®ćæ恌ä½æē”Ø恧恍悋
ęœ¬ē­‹ćØćÆ関äæ‚ćŖ恄恧恙恌、ćƒ¢ćƒ‡ćƒ«ć®čŖ¬ę˜Žć‚’ē”»é¢ć«äæå­˜ć§ćć‚‹ć®åœ°å‘³ć«å‡„ć„åŠ©ć‹ć‚‹ć‚„ć¤ć§ć™ć­...! Create Modelꖇ悂č‡Ŗå‹•ć§äæå­˜ć—ć¦ćć‚Œć¦ć„ćŸć‚Šć—ćŸć‚‰čØ€ć†ć“ćØćŖ恗ćŖę°—ćŒć—ć¾ć™。

GIS
- PostGISć§å®Ÿč£…ć•ć‚Œć¦ć„ć‚‹
- WKTćØGeoJSON (ShapeFilećÆå¤‰ę›ć™ć‚‹åæ…要恂悊)
- ē¾åœØćÆćƒ™ćƒ¼ćƒ€ć ćŒåÆč¦–åŒ–ć‚‚ć§ćć‚‹

Clusterd Tables
- ä»Šć¾ć§ćÆćƒ†ćƒ¼ćƒ–ćƒ«ć®åˆ†å‰²ćÆꙂ間č»øć ć£ćŸć®ć«åÆ¾ć—、ä»»ę„ć®ć‚«ćƒ©ćƒ ć§ć‚µćƒ–ćƒ†ćƒ¼ćƒ–ćƒ«ć«åˆ†ć‘ć‚‹(ćƒ‘ćƒ¼ćƒ†ć‚£ć‚·ćƒ§ćƒ³ć®ć‚¤ćƒ”ćƒ¼ć‚ø)

# Cloud TPU by kazunori_279
TPU
- John Hennessy:ć‚½ćƒ•ćƒˆć‚¦ć‚§ć‚¢ć ć‘ć§č§£ę±ŗć§ćć‚‹ę™‚ä»£ćÆēµ‚ć‚ć‚Šć¤ć¤ć‚ć‚‹ → Domain Specific Hardware恌åæ…要
- 実ćÆBigQueryćÆHardware Acceleration悒ä½æć£ć¦ć„ć‚‹ć®ć§é€Ÿć„
- GPUćÆé›»åŠ›ć‚’é£Ÿć†、DCć®č²»ē”Ø恮1/3~1/2ćÆ電力代ćØ悂čØ€ć‚ć‚Œć¦ć„ć‚‹。恓悌恌ē†ē”±ć§ę±äŗ¬ć®DCćÆGPU悒ē½®ć‹ć›ć¦ćć‚ŒćŖć‹ć£ćŸć‚Šć™ć‚‹
- GPU悂äø¦åˆ—ćØćÆ恄恈CPUćØåŒę§˜ć®å‡¦ē†ćŒå‹•ć„恦恄悋
- TPUćÆćƒ”ćƒ¢ćƒŖćøć®ć‚¢ć‚Æć‚»ć‚¹å›žę•°ć‚’ęø›ć‚‰ć™ć“ćØć§ę€§čƒ½ćŠć‚ˆć³é›»åŠ›ę¶ˆč²»é‡ć‚’å¤§å¹…ć«ę”¹å–„ć—ć¦ć„ć‚‹
- Cloud TPU v2 ć®ę–™é‡‘ä½“ē³»ćÆNormal ćØPreemptible(å„Ŗ先åŗ¦ä½Ž)ć§å€¤ę®µå·®ćŒć‚ć‚‹
- DAWNBenchć§ćƒ™ćƒ³ćƒćƒžćƒ¼ć‚ÆćŠć‚ˆć³ć‚³ć‚¹ćƒˆćƒ‘ćƒ•ć‚©ćƒ¼ćƒžćƒ³ć‚¹ćŒå…¬é–‹ć•ć‚Œć¦ć„ć‚‹

TPU Pod
- MPIćƒ©ć‚¤ć‚ÆćŖ悂恮悒ä½æć£ć¦TPUåŒå£«ć§é€šäæ”恗恦恄悋(all reduce)
ä»Šć¾ć§ćÆParameter ServerćØNodećÆgRPC恧通äæ”恗恦恄悋
- eBay恮äŗ‹ä¾‹ć§ćÆaccuracy +10%, 100x faster。č©¦č”ŒéŒÆčŖ¤ć®å›žę•°ć‚‚å¢—ćˆć‚‹ć®ć§accuracyć‚‚å‘äøŠ

# Auto ML by shuhei_fujiwara
Auto ML Vision
- Cloud Storage恫ē”»åƒć‚’ć‚¢ćƒƒćƒ—ćƒ­ćƒ¼ćƒ‰ć—、csv(path, label)悒ē”Øꄏ恗、å®Ÿč”Œć™ć‚‹ćØWebAPI恌ē”Ÿćˆć¦ćć‚‹

AutoML Natural Language
- CSVć‚’ć‚¢ćƒƒćƒ—ćƒ­ćƒ¼ćƒ‰ć™ć‚‹(ꖇę›ø, ć‚«ćƒ†ć‚“ćƒŖ)

AutoML Translation
TMXćƒ•ć‚”ć‚¤ćƒ«ć‚’ć‚¢ćƒƒćƒ—ćƒ­ćƒ¼ćƒ‰ć™ć‚‹
English→JapnesećÆę—¢ć«ć‚ć‚‹ćŒ、逆ćÆć¾ć 

å­¦ēæ’å¾Œć«å­¦ēæ’過ēØ‹ć®é·ē§»ć‚„ConfusionMatrix、é–“é•ć£ćŸéƒØ分ćŖ恩悂č”Øē¤ŗć—ć¦ćć‚Œć‚‹
- AutoMLć®ćƒ¢ćƒ‡ćƒ«ćƒ€ć‚¦ćƒ³ćƒ­ćƒ¼ćƒ‰ćÆꙂꜟ恌ęœŖå®šć ćŒć„ćšć‚ŒåÆčƒ½ć«ćŖ悋äŗˆå®š

ćƒ¢ćƒ‡ćƒ«ć®ä½œęˆć®ä¾”ę ¼
- Auto ML Vision (20$/per)
- AutoML Natural Language (3$/per)
- AutoML Translation (76$/per)
VisionćÆNASNetćŒćƒ™ćƒ¼ć‚¹ćØćŖć£ć¦ć„ć‚‹

~~~

ꬔ恫、BQMLćØAster恮ē°”単ćŖé•ć„ć«ć¤ć„ć¦č¦‹ć¦ć„ććŸć„ćØę€ć„ć¾ć™。Teradatać®č£½å“ć§ć‚ć‚‹Aster悂BQMLćØåŒę§˜ć«SQLć§ę©Ÿę¢°å­¦ēæ’ć‚’č”Œćˆć¾ć™。ē¾åœØBQMLćÆē·šå½¢å›žåø°ćØ惭ć‚øć‚¹ćƒ†ć‚£ćƒƒć‚Æ回åø°ć®2ć¤ć®ć‚ˆć†ć§ć™ćŒ、AsterćÆćŖćŖćŖ悓ćØ100ēخ锞仄äøŠć‚‚ć®é–¢ę•°ćŒē”Øę„ć•ć‚Œć¦ć„ć¦、NeuralNet悄XGBoostćŖć‚“ć‹ć‚‚ć‚µćƒćƒ¼ćƒˆć—ć¦ć„ć¾ć™。å„é–¢ę•°ćÆJava恧ę›øć‹ć‚Œć¦ć„ć‚‹ć®ć§ć™ćŒ、č¶³ć‚ŠćŖć„é–¢ę•°ćŒć‚ć‚Œć°č‡Ŗåˆ†ć§ę›ø恄恦UDFćØć—ć¦ć‚¤ćƒ³ć‚¹ćƒˆćƒ¼ćƒ«ć§ćć‚‹ć®ćŒå€‹äŗŗēš„恫ćÆę°—ć«å…„ć£ć¦ć„ć¾ć™。äø”č€…ć‚’ć‚·ćƒ³ć‚æ惃ć‚Æć‚¹ćƒ¬ćƒ™ćƒ«ć§äø¦ć¹ć‚‹ćØ仄äø‹ć®ć‚ˆć†ć«ćŖć‚Šć¾ć™。Aster偓ćÆč¦‹ę…£ć‚Œć¦ć„ćŖ恄ćØć‚„ć‚„å„‡å¦™ćŖå°č±”ć‚’å—ć‘ć¾ć™ć­。BQMLćÆć‚·ćƒ³ć‚æ惃ć‚Æć‚¹ć‹ć‚‰ęƒ³åƒć™ć‚‹é™ć‚Š、čŖ¬ę˜Žå¤‰ę•°Numeric/CategoryćÆč‡Ŗå‹•ć§åˆ¤åˆ„ć—ć¦ćć‚Œć¦ć„ć‚‹ć®ć§ć—ć‚‡ć†ć‹...?悂恗恝恆恠ćØ恗恟悉č‰Æ恄恧恙恭〜。

-- BQML (Train)
CREATE  `bqml_model` 
OPTIONS(model_type='logistic_reg') AS
SELECT
  label,
  os,
  is_mobile,
  country,
  pageviews
FROM
  `bigquery-public-data.google_analytics_sample.ga_sessions_*`;

-- BQML (Predict)
SELECT  *
FROM
  ml.EVALUATE(MODEL `bqml_codelab.sample_model`, (
SELECT
  label,
  os,
  is_mobile,
  country,
  pageviews
FROM
  `bigquery-public-data.google_analytics_sample.ga_sessions_*`;

-- Aster (Train)
SELECT * FROM GLM (
  ON (SELECT 1)
  PARTITION BY 1
  InputTable ('aster_train')
  OutputTable ('aster_model')
  InputColumns ('pageviews')
  CategoricalColumns ('os', 'is_mobile', 'county')
);

-- Aster (Predict)
 SELECT * FROM GLMPredict (
  ON aster_test
  ModelTable ('aster_model'')
  Family ('LOGISTIC')
  Link ('LOGIT')
);

ćƒŸćƒ¼ćƒˆć‚¢ćƒƒćƒ—ć§ē“¹ä»‹ć•ć‚Œć¦ć„ćŸćƒŖćƒ³ć‚ÆćÆć“ć‚Œć‹ć‚‰č¦‹ć¦ć„ćć¤ć‚‚ć‚Šć§ć™。
GCPUG http://gcpug.jp/
Google Cloud Platformć®ćƒŽć‚¦ćƒć‚¦ https://github.com/gcpug/nouhau
Google Cloud Next 2018 ć¾ćØ悁 https://github.com/gcpug/nouhau/issues/60

Aug 13, 2018

Vulture, Syndrome


ēœŸå±±ä»ć•ć‚“恮「ć‚·ćƒ³ćƒ‰ćƒ­ćƒ¼ćƒ 」悒čŖ­ćæēµ‚ć‚ć‚Šć¾ć—ćŸ。ć‚æć‚¤ćƒˆćƒ«ćÆćƒćƒ£ć‚¤ćƒŠ・ć‚·ćƒ³ćƒ‰ćƒ­ćƒ¼ćƒ ć«ć‹ć‘ć¦ć„ć‚‹ć®ć‹ćŖćØęƒ³åƒć—ć¦ć„ć¾ć™。ä¹…ć—ć¶ć‚Šć®ć‚¹ćƒ”ćƒ³ć‚Ŗ惕恧ćÆćŖć„ęœ¬ē·ØćØć‚ć£ć¦ę€ć‚ćšäø€ę°—čŖ­ćæć—ć¾ć—ćŸ。内容ćÆę±ę—„ęœ¬å¤§éœ‡ē½ć‚’ęć„ćŸć‚‚ć®ć§、首都電力ćØć„ć†ę±äŗ¬é›»åŠ›ć‚’ęØ”ć—ćŸä¼šē¤¾ćŒå‡ŗć¦ćć¾ć™。äŗ‹å®ŸćØč™šé£¾ćŒå…„ć‚Šę··ć˜ć£ć¦ć„ć‚‹ć‚“ć ćØćÆę€ć„ć¾ć™ćŒ、ę”æåŗœć®éˆć•ć‚„é¦–éƒ½é›»ä¼šé•·ć®ē„”č²¬ä»»ć•ćÆ見恦恄恦複雑ćŖę°—ęŒć”ć«ćŖć‚Šć¾ć™。ćć®ćć‚‰ć„ćƒŖć‚¢ćƒ«ć ćØ恄恆恓ćØ恧恗悇恆恌....。äŗŗåć«ć¤ć„ć¦ćÆå½“ę™‚ć®ę–¹ć€…ćØē…§ć‚‰ć—åˆć‚ć›ć‚‹ćØ、仄äø‹ć®ć‚ˆć†ć«ćŖ悋恧恗悇恆恋。

ęæ±å°¾é‡č‡£→勝äæ£ę’ä¹…
č©ęœ¬ć‚ć‹ć­→äøøå±±ę”‚é‡Œå„ˆ
äø²ę‘勝之→吉ē”°ę˜ŒéƒŽ
å¤č°·å…‰å¤Ŗ郎→菅ē›“äŗŗ
갓野ē¾©äæ”→ꞝ野å¹øē”·
佐ä¼Æꘇ→鉢呂吉雄

åŒć˜ćēœŸå±±ć•ć‚“ć®åŽŸē™ŗäŗ‹ę•…ć«é–¢ć™ć‚‹å°čŖ¬ć®ćƒ™ć‚¤ć‚øćƒ³ć‚‚čŖ­ć‚“恠恓ćØćŒć‚ć‚Šć¾ć™ćŒ、ē¦å³¶åŽŸē™ŗäŗ‹ę•…恮恂ćØ恫å‡ŗćŸęœ¬ć ćØę€ć£ć¦ć„ćŸć®ć§、å‡ŗē‰ˆć•ć‚ŒćŸå¹“恌2008å¹“ć ć£ćŸćØ恄恆恓ćØć«é©šćć¾ć—ćŸ。ćŠćŠć¾ć‹ćŖ恂悉恙恘ćÆ北äŗ¬äŗ”č¼Ŗć®é–‹ä¼šå¼ć«å‘ć‘ć¦ēŖč²«å·„äŗ‹ć§é€²ć‚ćŸåŽŸē™ŗćŒå¤§åœ°éœ‡ćØćƒ”ćƒ«ćƒˆćƒ€ć‚¦ćƒ³ć‚’čµ·ć“ć™ćØć„ć£ćŸć‚‚ć®ć§、ę±ę—„ęœ¬å¤§åœ°éœ‡ä»„å‰ć§ć‚ć‚Œć°ćć‚“ćŖ恓ćØčµ·ćć‚‹ćÆ恚ćŖ恄!ćØäø€č¹“ć§ćć¦ć„ćŸć‹ć‚‚ć—ć‚ŒćŖć„å†…å®¹ć§ć™。
ć©ć”ć‚‰ć®ęœ¬ć‚‚äŗ‹å®Ÿć®ćæ恧ćŖ恏、å°‘ć—å¼•ć„ćŸē›®ē·šć‹ć‚‰åŽŸē™ŗäŗ‹ę•…ć®ęµć‚Œć‚„ćć“ć§åƒć„ć¦ć„ćŸć‚‚ć—ććÆåƒć„ć¦ć„ć‚‹ę–¹ć€…ć«ć¤ć„ć¦ć‚‚ć†1åŗ¦č€ƒćˆć¦ćæ恟恄äŗŗć«ćŠå‹§ć‚ć—ćŸć„ęœ¬ć§ć™。

Aug 6, 2018

Functional Programming in Java



4幓čæ‘ćå‰ć«å‡ŗē‰ˆć•ć‚ŒćŸęœ¬ć§ć™ćŒ、Javać«ć‚ˆć‚‹é–¢ę•°åž‹ćƒ—ćƒ­ć‚°ćƒ©ćƒŸćƒ³ć‚°ć‚’čŖ­ć‚“恧ćæć¾ć—ćŸ。ć‚æć‚¤ćƒˆćƒ«ć‚’ćć®ć¾ć¾å—ć‘å–ć‚‹ćØé–¢ę•°åž‹ć«čˆˆå‘³ć®ć‚ć‚‹äŗŗ恗恋čŖ­ć¾ćŖ恕恝恆恧恙恌、é…å»¶č©•ä¾”、ęœ«å°¾å‘¼ć³å‡ŗć—ęœ€é©åŒ–、ćƒ”ćƒ¢åŒ–ćŖć©ć®č©±ć‚‚č¼‰ć£ć¦ć„ć¦、å…Ø体ēš„ć«ę€§čƒ½å‘äøŠć«ē¹‹ćŒć‚‹č¦ē“ ćŒć”ć‚Šć°ć‚ć‚‰ć‚Œć¦ć„ć¾ć™。ęœ«å°¾ć®ä»˜éŒ²ć«DRYåŽŸå‰‡ćŒč¼‰ć£ć¦ć„ć‚‹ć®ć§ć™ćŒ、ćć“ć«"DRY恮ē²¾ē„žć®å‰‡ć£ć¦、恓恓恧ćÆćć®å†…å®¹ć‚’čŖ¬ę˜Žć—ć¾ć›ć‚“。URL恮čؘäŗ‹ć‚’参ē…§ć—ć¦ćć ć•ć„"ćØę›øć„ć¦ć‚ć‚‹ć®ćŒć”ć‚‡ć£ćØ面ē™½ć‹ć£ćŸć§ć™。

Jul 17, 2018

Money Never Sleeps


čØ€ć®č‘‰ć®åŗ­ćØåŒć˜ćć‚‰ć„ä½•åŗ¦ć‹č¦³ć¦ć„ć‚‹Wall Street: Money Never Sleepsć‚’č¦³ć¾ć—ćŸ。äø€ē•Ŗęœ€åˆć«č¦³ćŸć‚­ćƒƒć‚«ć‚±ćÆēˆ¶ćŒå®¶ć§č¦³ć¦ć„ćŸć“ćØ恧恙恌、ćć®å¾Œ ēœŸå±±ä»ć•ć‚“ć®ćƒć‚²ć‚æć‚«ć‚·ćƒŖćƒ¼ć‚ŗ悒čŖ­ć‚“ć§é‡‘čžć«čˆˆå‘³ć‚’ęŒć£ć¦ć‹ć‚‰2回ē›®ä»„é™ć‚’č¦³ć‚‹ć‚ˆć†ć«ćŖć‚Šć¾ć—ćŸ。ćƒ¬ćƒ“ćƒ„ćƒ¼ć§ćÆå‰ä½œć®ę–¹ćŒč‰Æć‹ć£ćŸćØ恄恆äŗŗćŒå¤šć„ćæ恟恄恧恙恌、前作ćÆęœŖ恠恫ꉋ悒å‡ŗć—ć¦ć„ć¾ć›ć‚“。å…ˆć«ę–°ć—ć„ę–¹ć‚’č¦³ć‚‹ćØē”»č³Ŗ恌äø‹ćŒć‚‹ć®ćŒå«Œć ćŖ恁ćØ適当ćŖē†ē”±ć‚’恤恑恦éæć‘ć¦ć„ćŸć‚“ć§ć™ćŒ、čŖæć¹ć¦ćæ恟ćØć“ć‚å‰ä½œćØ2å¹“ć—ć‹ēµŒć£ć¦ćŖ恄悈恆ćŖ恮恧今åŗ¦č¦³ć¦ćæ悈恆ćØę€ć„ć¾ć™。
ćƒ©ć‚¹ćƒˆć«ęµć‚Œć‚‹Talking Heads恮This Must Be The PlacećÆ何åŗ¦č¦³ć¦ć‚‚ć‚„ć£ć±ć‚Šå„½ćć§、ę®µć€…ć‚«ćƒ”ćƒ©ćŒäøŠć‚’å‘ćć«é€£ć‚Œć¦ćŠć˜ć•ć‚“ć‚‚ć¾ćŸä»•äŗ‹é ‘å¼µć‚‰ćŖ恂恋悓ćŖćØć„ć†ę°—ęŒć”ć«ćŖć‚Šć¾ć™。ć“ć®ę›²ćÆMarvel恮Agents of S.H.I.E.L.D恫悂ä½æć‚ć‚Œć¦ć„ć‚‹ćæ恟恄恧恙恭。å…Ø恏違恆ć‚øćƒ£ćƒ³ćƒ«ćŖć®ć§é•å’Œę„ŸćŒć‚ć£ć¦é¢ē™½ć„恧恙。CDē‰ˆć®éŸ³ęŗć‚‚悂恔悍悓č‰Æ恄恧恙恌、Youtubeć«ć‚ćŒć£ć¦ć„ć‚‹Naive Melodyē‰ˆćÆé›°å›²ę°—ć‚‚ć‚ć„ć¾ć£ć¦ęœ€é«˜ć§ć™。(ć“ć®å…ƒć®å‹•ē”»ćÆć©ć“ć§č²·ćˆć‚‹ć®ć ć‚ć†!)

Jul 5, 2018

Hi, YARN-6640

YARNäøŠć§å‹•ć„恦恄悋、ćØć‚ć‚‹ć‚¢ćƒ—ćƒŖ恌YARNć®ćƒć‚°ć‚’čøćæęŠœćć¾ć—ćŸ。
AM heartbeat stuck when responseId overflows MAX_INT

恓悌ćÆå…ø型ēš„ćŖć‚Ŗćƒ¼ćƒćƒ¼ćƒ•ćƒ­ćƒ¼ć®ćƒć‚°ć§、Application ManagerćØResource Manager間恮悄悊ćØ悊悒intåž‹ć®å¤‰ę•°ć§åæœē­”恮ID悒ē®”ē†ć—ęÆŽå›žć‚¤ćƒ³ć‚ÆćƒŖćƒ”ćƒ³ćƒˆć—ć¦ć„ć‚‹ć®ć§ć™ćŒ、ćć®å€¤ćŒ2147483647恫達恙悋ćØ、ę¬”å›žć®ID恌-2147483648恫ćŖ悊、å‰å›žć®ćƒ¬ć‚¹ćƒćƒ³ć‚¹ćØć®ę•“åˆę€§ćŒå–ć‚Œć¦ć„ćŖ恄ćØ恆恓ćØć§ä¾‹å¤–ć‚’ęŠ•ć’ć¾ć™。YARN恮2.8.2恧äæ®ę­£ć•ć‚Œć¦ć„ć¾ć™ćŒ、ćć‚Œä»„å‰ć®ćƒćƒ¼ć‚øćƒ§ćƒ³ć‚’ä½æć£ć¦ćŠć‚Š、ćƒćƒ¼ćƒˆćƒ“ćƒ¼ćƒˆć®é–“éš”ćŒēŸ­ć„ć‚¢ćƒ—ćƒŖć‚’ćƒ­ćƒ³ć‚°ćƒ©ćƒ³ć—ć¦ć„ć‚‹ć‚ˆć†ćŖå “åˆć«ćÆäø€åŗ¦ē¢ŗčŖć—恦ćæ悋ćØč‰Æć„ć‹ć‚‚ć—ć‚Œć¾ć›ć‚“。
ćŖ恊、AM-RM間恠恑恧ćŖ恏Name Manager-Resource Mangeré–“ć§ć‚‚åŒę§˜ć®ćƒ­ć‚ø惃ć‚ÆćÆå«ć¾ć‚Œć¦ć„ć¾ć—ćŸ。恓恔悉ćÆ2.8.4恧äæ®ę­£ć•ć‚Œć¦ć„ć¾ć™。
NM heartbeat stuck when responseId overflows MAX_INT

Jul 4, 2018

Visualize heatmap on Aster AppCenter

Aster AppCenter恧heatmap恮Custom Visualizationć‚’ä½œć£ć¦ćæć¾ć—ćŸ。
https://github.com/ebyhr/Teradata/tree/master/aac_heatmap



ä½æć„ę–¹ćÆdb_table_nameć«ćƒ†ćƒ¼ćƒ–ćƒ«åć‚’ęŒ‡å®šć—ć¦、vizType恫heatmapć‚’ęŒ‡å®šć—ć¾ć™。
heatmapć®ćƒ©ć‚¤ćƒ–ćƒ©ćƒŖćÆåŗ§ęØ™ć‚’x, yęŒ‡å®šć™ć‚‹ć‚ˆć†ćŖćƒ‡ćƒ¼ć‚æć®ęŒć”ę–¹ć‚’å‰ęć«ć—ć¦ć„ć‚‹ć‚‚ć®ćŒå¤šć„ć§ć™ćŒ、ć‚ćˆć¦ćƒ†ćƒ¼ćƒ–ćƒ«ć‚’ćć®ć¾ć¾č”Øē¤ŗć—ć¦č‰²ä»˜ć‘ć™ć‚‹ć‚ˆć†ć«ć—ć¦ć„ć¾ć™。
insert into app_center_visualizations (json) values (
'{
    "db_table_name": "public.confusionmatrix",
    "vizType": "heatmap",
    "title": "Report by confusionmatrix"
}')

Teradata AppCenterćÆćŸć¾ć«č§¦ć£ć¦ć„ć¾ć™ćŒ、UXćŒå¾®å¦™ć™ćŽć¦ć‚ć¾ć‚Šä½æć†ę°—ćŒčµ·ćć¾ć›ć‚“。惐惃ć‚Æć‚Øćƒ³ćƒ‰ć«Kubernetesć‚’ę®ćˆćŸć®ćÆå°†ę„ēš„恫ćÆć„ć„åˆ¤ę–­ć«ćŖ悋ćØꀝ恆悓恧恙恌、ē¾ę™‚ē‚¹ć§ćÆć‚Ŗćƒ¼ćƒćƒ¼ć‚Øćƒ³ć‚øćƒ‹ć‚¢ćƒŖćƒ³ć‚°ć‹ćŖ恁ćØę„Ÿć˜ć¾ć™。Javać§ć”ć‚‡ć£ćØć‚³ćƒ¼ćƒ‰ć‚’ę›ø恍恟恄恠恑ćŖ恮恫Dockerfileę›øćć®ćÆé¢å€’ć§ć™。

č©±ćŒå¤‰ć‚ć‚Šć¾ć™ćŒ、꜀čæ‘SQL悒äŗŒé‡å®Ÿč”Œć™ć‚‹ć‚ˆć†ćŖUDFć‚’ä½œć‚ć†ć‹ćŖćØč€ƒćˆć¦ć„ć¾ć™。åˆ†ęžć‚Æć‚ØćƒŖć§ę¬²ć—ć„å “é¢ćÆēš†ē„”恧恙恌、ćƒćƒƒćƒå‡¦ē†ć§ä½æć„ćŸć„å “é¢ćÆå¤šć€…ć‚ć‚Šć¾ć™。
Aster恮SQL-MRć§ć‚ć‚Œć°ć‚³ćƒ¼ćƒ‰ć‚’ę›øćć®ćÆ3åˆ†ć‚‚ć‚ć‚Œć°ēµ‚ć‚ć‚‹ć®ć§ć™ćŒ、č‰Æć„åå‰ćŒę€ć„ć¤ć‹ćŖć„ć®ć§ćƒšćƒ³ćƒ‡ć‚£ćƒ³ć‚°ć«ć—ć¦ć„ć¾ć™。
ä¾‹ćˆć°å…ØéƒØć®ćƒ†ćƒ¼ćƒ–ćƒ«ć‚’å‰Šé™¤ć—ćŸć„ę™‚ćÆ恓悓ćŖć‚Æć‚ØćƒŖ恫ćŖć‚Šć¾ć™。Java恧恂悌恰äø­ć®SELECTꖇ恮ēµęžœć‚’ResultSetć§å—ć‘å–ć‚Š、ćƒ«ćƒ¼ćƒ—ć‚’å›žć—ć¦executeQueryć‚’å®Ÿč”Œć™ć‚‹ć ć‘ć§ć™。
select * from xxx ( on (select 1)
 query ('select ''drop table '' || a.schemaname || b.tablename from nc_system.nc_all_schemas a inner join nc_system.nc_all_tables b on a.schemaid = b.schemaid')
)

Jul 2, 2018

Learning Coursera Deep Learning

Coursera恧deeplearning.ai恮Neural Networkć®ć‚³ćƒ¼ć‚¹ć‚’å—č¬›ć—å§‹ć‚ć¾ć—ćŸ。ć”ć‚‡ć†ć©åŒåƒšćŒć‚„ć£ć¦ć„ćŸć“ćØ悂恂悊、åˆ†ć‹ć‚‰ćŖ恄ćØ恓悍恌å‡ŗć¦ććŸę™‚ć«čžć‘ć‚‹ć®ćŒć ć„ć¶åŠ©ć‹ć£ć¦ć„ć¾ć™。Neural Networks and Deep LearningćÆēµ‚ć‚ć‚Š、今ćÆImproving Deep Neural Networks: Hyperparameter tuning, Regularization and Optimizationć‚’å—č¬›ć—ć¦ć„ć¾ć™。
åŒåƒšć‚‚åŒć˜ē¾č±”ć«é­é‡ć—ć¦ć„ć¾ć—ćŸćŒ、Jupyter Notebook恧Kernel恌ć‚Øćƒ©ćƒ¼ć«ćŖ悊、äæå­˜ć‚‚å‡ŗę„ćŖ恏ćŖć£ćŸę™‚ćÆ、å›ŗć¾ć£ćŸē”»é¢ćÆćć®ć¾ć¾ć®ēŠ¶ę…‹ć§、 Coursera恮ē”»é¢ć‚’刄ć‚æ惖恧開恍、Open Notebookć‚’ęŠ¼ć—ć¦2恤開恄恟ēŠ¶ę…‹ć«ć™ć‚‹ćØ、ć‚Øćƒ©ćƒ¼ć«ćŖć£ć¦ć„ćŸNotebookćŒå¾©ę“»ć™ć‚‹ćØę€ć„ć¾ć™。



é€±ęœ«ć«「天ē©ŗ恮ēŸ¢ćÆ恩恓ćø?」悒čŖ­ćæć¾ć—ćŸ。Wć‚·ćƒŖćƒ¼ć‚ŗćÆ10ęœˆć«ē™ŗ売äŗˆå®šć®「äŗŗé–“ć®ć‚ˆć†ć«ę³£ć„ćŸć®ć‹?」恌꜀ēµ‚å·»ćŖ恮恧恙恭。å¾…ć”é ć—ć„ćŖ恁ćØę›ø恓恆ćØć—ć¾ć—ćŸćŒ、꜀ēµ‚å·»ćŒå¾…ć”遠恗恄ćØ恄恆恮ćÆę—©ćēµ‚ć‚ć£ć¦ę¬²ć—ć„ćØć„ć†ć‚ˆć†ć«ć‚‚ćØ悉恈悋恓ćØćŒć§ćć¾ć™。SFćŒå¾—ę„ćŖNetflixć‚ćŸć‚Šć§ć‚¢ćƒ‹ćƒ”ć‹ćƒ‰ćƒ©ćƒžåŒ–ć—ć¦ćć‚Œć‚‹ćØå¬‰ć—ć„ć®ć§ć™ćŒ、åÆčƒ½ę€§ćÆä½Žć„ć ć‚ć†ćŖ恁...。

Jun 18, 2018

Read NEW ELITE

ćƒ‹ćƒ„ćƒ¼ć‚ØćƒŖćƒ¼ćƒˆ ć‚°ćƒ¼ć‚°ćƒ«ęµ・ę–°ć—ć„ä¾”å€¤ć‚’ē”Ÿćæå‡ŗ恗äø–ē•Œć‚’å¤‰ćˆć‚‹äŗŗćŸć”ćØć„ć†ęœ¬ć‚’čŖ­ć‚“恧ćæć¾ć—ćŸ。ēµč«–恋悉čØ€ć†ćØ、ć”ć‚‡ć£ćØå¾®å¦™ć‹ćŖ...ćØć„ć†ę„Ÿęƒ³ć§ć™。č‰²ć‚“ćŖ会ē¤¾ć‚„čµ·ę„­å®¶ć®ä¾‹ć‚’å‡ŗ恗過恎恦、č©±ćŒē™ŗę•£ć—ć¦ć„ć‚‹ć‚ˆć†ć«ę€ć„ć¾ć™。čŖ­ćæēµ‚ć‚ć£ć¦ēµå±€ć“ć®ęœ¬ćŒ1ē•Ŗä¼ćˆćŸć‹ć£ćŸć®ć‹ćÆä½•ć ć£ćŸć‚“ć ć‚ć†ćØ、č‹„å¹²ć‚‚ć‚„ć‚‚ć‚„ć£ćØć—ć¦ć—ć¾ć„ć¾ć—ćŸ。č¦ę‰€č¦ę‰€ć§ć“ć‚ŒćÆå‚č€ƒć«ćŖ悊恝恆ćØ恄恆éƒØ分ćÆć‚‚ć”ć‚ć‚“ć‚ć‚Šć¾ć™。
 å€‹äŗŗēš„恫ćÆć‚æćƒ¼ć‚²ćƒƒćƒˆćØćŖ悋čŖ­č€…ć®å±¤ćÆćƒžćƒćƒ¼ć‚øćƒ£ćƒ¼ä»„äøŠć‚„HRć®ę–¹ć€…ćŖå°č±”ć‚’å—ć‘ćŸć®ć§、ćć†ć„ć£ćŸę–¹ć€…ćŒčŖ­ć‚“恧ćæ悋ćØå…Øćé•ć†ę„Ÿęƒ³ć‚’ęŒćŸć‚Œć‚‹ć‚“ć ć‚ć†ćŖ恁ćØ。

Jun 14, 2018

Read Learned Indexes Paper

今ꛓćŖ恌悉A Machine Learning Approach to Databases Indexes悒čŖ­ć‚“恧ćæć¾ć—ćŸ。論ꖇ恮äø­čŗ«ćÆćƒ‡ćƒ¼ć‚æćƒ™ćƒ¼ć‚¹ć®ć‚¤ćƒ³ćƒ‡ćƒƒć‚Æć‚¹(ē“¢å¼•)ć«ę©Ÿę¢°å­¦ēæ’悒適ē”Ø恗恦ćæ悈恆ćØ恄恆悂恮恧恙。
ć‚¤ćƒ³ćƒ‡ćƒƒć‚Æć‚¹ć®ēخ锞ćÆå¤§ćć3ć¤ć«åˆ†ć‘ć¦čŖ¬ę˜Žć•ć‚Œć¦ć„ć¾ć™。

#1 Range Index
å¤šćć®DB恧ä½æć‚ć‚Œć¦ć„ć‚‹BęœØćØ恮ęÆ”č¼ƒćŒć•ć‚Œć¦ć„ć¾ć™。BęœØ恌f(key) ➝ posćØćƒ”ćƒ³ćƒć‚¤ćƒ³ćƒˆć§ć‚­ćƒ¼ćŒå­˜åœØć™ć‚‹ćƒšćƒ¼ć‚øć®é–‹å§‹ä½ē½®ć‚’č”Ø恙恮恫åÆ¾ć—ć¦、ćƒ‡ćƒ¼ć‚æćŒć‚ć‚‰ć‹ć˜ć‚ć‚½ćƒ¼ćƒˆć•ć‚Œć¦ć„ć‚‹å‰ęćØ恗恦Learned Ranged Index恧ćÆ[Ė†y − ∆, yĖ† + ∆] (∆ćÆmaximum error)恮ēÆ„å›²ć‚’ęŽ¢ć—ć«č”Œćć‚ˆć†ć§ć™。

#2 Point Index
Hash Mapć®ć‚ˆć†ćŖ1åƾ1ć®ćƒ‡ćƒ¼ć‚æå–å¾—ć«ć¤ć„ć¦ć§ć™。O(1)ćÆę”¹å–„ć®ä½™åœ°ćŒćŖć„ć‚ˆć†ć«č¦‹ćˆć¾ć™ćŒ、ćƒćƒƒć‚·ćƒ„ćŒč”ēŖć™ć‚‹å “合Linked Listć‚’čµ°ęŸ»ć™ć‚‹ć®ć«ę™‚é–“ćŒć‹ć‹ć‚‹ć®ć§、ćć‚Œć‚’ę”¹å–„ć™ć‚‹ćØć„ć†ć®ćŒäø»ćŖē‹™ć„ć®ć‚ˆć†ć§ć™。#1恧ē“¹ä»‹ć—ćŸf悒CDF (Cumulative Distribution Function)ćØ恗恦、h(x) = mCDF(x)ć§ćƒćƒƒć‚·ćƒ„ć‚’č؈ē®—ć—ć¾ć™。恓恆恙悋恓ćØć§č”ēŖć®ćŖć„ćƒćƒƒć‚·ćƒ„å€¤ć‚’ē®—å‡ŗ恗恦恄悋ćØ恮恓ćØ恧恙。

#3 Existence Index
ē‰¹å®šć®ć‚­ćƒ¼ćŒćƒ‡ćƒ¼ć‚æć«å­˜åœØć—ć¦ć„ć‚‹ć‹ć‚’åˆ¤ę–­ć™ć‚‹Index恫恤恄恦恧恙。Binary Classificationć‚’ęƒ³å®šć—、1/0ć®ćƒ©ćƒ™ćƒ«ć‚’ęŒć£ćŸćƒ‡ćƒ¼ć‚æ恫åÆ¾ć—ć¦RNN/CNN & Sigmoidć§åˆ†é”žć—、ēµęžœćŒYesćŖć‚‰ćć®ć¾ć¾、NoćŖ悉Bloom Filter悒通恗恦YesćøćØć„ć†ęµć‚Œć®ć‚ˆć†ć§ć™。Bloom Filter恌False Negative悒起恓恕ćŖ恄恓ćØć‚’ę“»ē”Øć—ć¦ć‚‹ć‚ˆć†ć§ć™。

ć‚ˆć‚Šč©³ć—ć„č«–ę–‡ćŒThe Case for Learned Index Structures恧、ć‚°ćƒ©ćƒ•ćØć‚³ćƒ¼ćƒ‰ćŒå°‘ć—å¢—ćˆć¦ć„ć‚‹ć®ć§、恓恔悉悂čŖ­ć‚€ćØē†č§£ć—ć‚„ć™ć„ć‹ć‚‚ć—ć‚Œć¾ć›ć‚“。

May 3, 2018

Robot in Shinagawa Prince Hotel

NHK Worldć‚’č¦‹ć¦ćŸć‚‰å“å·ć®ćƒ—ćƒŖćƒ³ć‚¹ćƒ›ćƒ†ćƒ«ć«ć„ć‚‹ćƒ‡ćƒŖ惐ćƒŖćƒ¼ćƒ­ćƒœćƒƒćƒˆćŒå‡ŗę¼”ć—ć¦ć¾ć—ćŸ。å¾“ę„­å“”ć‹ć‚‰ćŠę°“ć‚’å—ć‘å–ć£ćŸć‚‰č‡Ŗåˆ†ć§ć‚Øćƒ¬ćƒ™ćƒ¼ć‚æ恫恮悊、å®æę³Šč€…ć®éƒØå±‹ć¾ć§é‹ć‚“ć§ćć‚Œć‚‹ć‚ˆć†ć§ć™。
Ever-Evolving Shinagawa


Yiingyać•ć‚“ćŒćƒ­ćƒœćƒƒćƒˆć®č©•ä¾”ć«ę˜Ÿ5恤悒éøꊞ恙悋ćØ「ć‚„ć£ćŸć〜!」ćØē¬‘锔悒č”Øē¤ŗ恗ćŖćŒć‚‰å·¦å³ć«ęŗć‚Œć¦ć„ć‚‹ć®ćŒćŖ悓ćØ悂åÆꄛ恄!名前ćÆHARRYćØ恄恆ćæ恟恄恧恙。ē”Ÿć§č¦‹ć¦ćæ恟恄ćŖ恁〜šŸ¤–
http://www.princehotels.co.jp/shinagawa/ntower/robot.html

May 2, 2018

SQL as a Visualization


GWäø­ć«ćƒ’ćƒžć ć£ćŸć®ć§ä½œć£ć¦ć„ć‚‹ćƒ—ćƒBIćƒ„ćƒ¼ćƒ«ć§ć™。ć¾ć ć¾ć ä½œć‚Šé€”äø­ćŖ悓悓恧恙恌、SQL悒ę›øć„ć¦ćć®ć¾ć¾åÆč¦–åŒ–ć§ćć‚‹ć®ć§ä½•ę°—ć«ä¾æåˆ©ć§ć™。ć‚³ćƒ”ćƒ³ćƒˆć®éƒØåˆ†ć®--visualize=X恧恩悓ćŖć‚°ćƒ©ćƒ•ć‚’č”Øē¤ŗć™ć‚‹ć‹ć‚’ęŒ‡å®šć—、ćć®å¾Œć‚ć§åæ…要ćŖćƒ‘ćƒ©ćƒ”ćƒ¼ć‚æć‚’å…„åŠ›ć—ć¾ć™。今恮ćØ恓悍ćÆē“ ć®ćƒ†ćƒ¼ćƒ–ćƒ«、ę£’ć‚°ćƒ©ćƒ•、ꊘ悌ē·šć‚°ćƒ©ćƒ•、ćƒ‘ć‚¤ćƒćƒ£ćƒ¼ćƒˆ、ćƒćƒ–ćƒ«ćƒćƒ£ćƒ¼ćƒˆ、ć‚²ćƒ¼ć‚øćƒćƒ£ćƒ¼ćƒˆćÆåÆč¦–åŒ–ć§ćć‚‹ć‚ˆć†ć«ć—ć¦ć‚ć‚Šć¾ć™。

--visualize=line,x=custkey,y=sumprice
select 
 custkey
 ,sum(totalprice) as sumprice
from tpch.tiny.orders
group by 1
order by 1
;

äø­čŗ«ćØ恗恦ćÆćƒ•ćƒ¬ćƒ¼ćƒ ćƒÆćƒ¼ć‚ÆćÆRailsć§ć‚°ćƒ©ćƒ•ć®éƒØ分ćÆHighchartć«ć—ć¾ć—ćŸ。ęœ€åˆćÆRuby恧ē°”å˜ć«ć‚°ćƒ©ćƒ•ć‚’å‡ŗ恛悋Chartkick悒ä½æć£ć¦ć„ćŸć‚“ć§ć™ćŒ、ꏏē”»ć§ćć‚‹ēØ®é”žćŒé™ć‚‰ć‚Œć¦ć„ć‚‹ć®ć§、Highchartć«å¤‰ę›“ć—ć¦ć„ć¾ć™。

実運ē”Øć‚„åˆ©ä¾æꀧ悒考恈悋ćØ、仄äø‹ć®ć‚ˆć†ćŖćƒć‚¤ćƒ³ćƒˆć‚’å®Ÿč£…ć—ć¦ć„ćåæ…č¦ćŒć‚ć‚‹ć‹ćŖćØč€ƒćˆć¦ć„ć¾ć™。
  • ć‚¢ć‚Æć‚»ć‚¹ęØ©ć®åˆ¶å¾”
  • ć‚Æć‚ØćƒŖć®å®Ÿč”Œćƒ¦ćƒ¼ć‚¶ćƒ¼ć®åˆ¶å¾”
  • ć‚Æć‚ØćƒŖē·Ø集Ꙃ恮悵ć‚øć‚§ć‚¹ćƒćƒ§ćƒ³
  • ćƒ‘ćƒ©ćƒ”ćƒ¼ć‚æ悒ä½æē”Øć—ćŸå®Ÿč”Œ
  • ć‚¹ć‚±ć‚øćƒ„ćƒ¼ćƒŖćƒ³ć‚°å®Ÿč”Œ
  • ēµęžœć®ćƒ”ćƒ¼ćƒ«é…äæ”

Did you cry in doutor?

ęÆŽå¹“å†¬ć«ćŖ悋ćØé¼»ć„ć¾ć‚ŠćŒč¾›ć™ćŽć‚‹ć®ć§、ę—„åø°ć‚Šć§é¼»äø­éš”ę¹¾ę›²ē—‡ć®ę‰‹č”“ć‚’å—ć‘ć¦ćć¾ć—ćŸ。恓悌恧čŖ˜ę‹ć•ć‚Œć¦å£ć«ć‚¬ćƒ ćƒ†ćƒ¼ćƒ—ć‚’č²¼ć‚‰ć‚Œć‚‹ć‚ˆć†ćŖäŗ‹ę…‹ć«ćŖć£ć¦ć‚‚ēŖ’ęÆ恗ćŖ恄恧恙ćæć¾ć™。
ć‚¬ćƒ¼ć‚¼ć‚’ęŠœćēž¬é–“ćŒē›ø当ē—›ć„ćØ恄恆恮ćÆå™‚ć§čžć„ć¦ć„ćŸć‚“ć§ć™ćŒ、éŗ»é…”恮ēž¬é–“ćŒ1ē•Ŗē—›ć‹ć£ćŸć§ć™。é¼»ć®å„„ć«ę³Øå°„ć™ć‚‹ć‚“ć§ć™ćŒ、é””é¢ć®éŖØ恫ē›“ꎄę³Øå°„ć—ć¦ć„ć‚‹ć‚ˆć†ćŖę„Ÿč¦šć§é””ćŒćƒ”ć‚­ćƒ”ć‚­ć„ć£ć¦ć¾ć—ćŸ。完å…Øć«ćƒŽćƒ¼ćƒžćƒ¼ć‚Æć ć£ćŸć®ć§ć‚ć¾ć‚Šć«ē—›ć•ć«ę”¾åæƒēŠ¶ę…‹ć«。ć‚µćƒƒć‚«ćƒ¼ć«ä¾‹ćˆć‚‹ćØ、ę ¼äø‹ć ćØę€ć£ć¦ćŸćƒćƒ¼ćƒ ć‹ć‚‰č©¦åˆé–‹å§‹å‰ć«ēŖē„¶ćƒœćƒ‡ć‚£ćƒ–ćƒ­ćƒ¼ć‚’é£Ÿć‚‰ć†ćæ恟恄ćŖēŠ¶ę³ć§ć™。恂ćØéŗ»é…”薬恫åæƒę‹ę•°ć‚’ć‚ć’ć‚‹ä½œē”Ø悂恂悋悈恆恧、ę³Øå°„ć‚’ć•ć‚ŒćŸć ć‘ćŖ恮恫å…Ø力ē–¾čµ°ć—ćŸć‚ćØćæćŸć„ć«čƒøćŒć©ćć©ćć—ćŸć®ć«ćÆē„¦ć‚Šć¾ć—ćŸ。ę‰‹č”“ćŒå§‹ć¾ć£ćŸć‚ćØćÆ当ē„¶ē—›ćæćÆćŖ恏、ä»£ć‚ć‚Šć«ćƒŽćƒŸć§ć‚“ćƒ³ć‚“ćƒ³ć‚„ć‚‰ć‚ŒćŸć‚Š、éŖØ悒ć‚ø惧悭ć‚ø惧悭恙悋悈恆ćŖäøåæ«ćŖę„Ÿč§¦ćØć®ęˆ¦ć„ć§ć™。途äø­ć”ć‚‡ć£ćØę°—ęŒć”ę‚Ŗ恏ćŖć£ć¦ć—ć¾ć„、1, 2åˆ†ä¼‘ę†©ć‚’ęŒŸć‚“ć§ć‚‚ć‚‰ć„ć¾ć—ćŸ。ēµ‚ć‚ć£ćŸć‚ćØć«å‰Šć‚Šå–ć£ćŸć‚ćØ恮éŖØć‚’č¦‹ć›ć¦ć‚‚ć‚‰ć£ćŸć‚“ć§ć™ćŒ、ć„ć‚„ć„ć‚„å–ć‚ŠéŽćŽć‚„ć‚...ćØć„ć†ćć‚‰ć„ć®é‡ć§ć—ćŸ。č”“å¾ŒćÆ1ę™‚é–“ć»ć©ćƒ™ćƒƒćƒ‰ć§ä¼‘ę†©ć—ćŸå¾Œć«é€€é™¢ć§ć™。
č”“å¾Œć®ē—›ćæč‡Ŗ体ćÆ恝悓ćŖ恫ē„”恄悓恧恙恌、口呼åø恗恋恧恍ćŖ恄ćØć„ć†ć®ćŒć‹ćŖć‚Šć‚¹ćƒˆćƒ¬ć‚¹ć§ćÆć‚ć‚Šć¾ć™。꜀ēµ‚ēš„恫ćÆę…£ć‚Œć¾ć—ćŸćŒ、ę‰‹č”“å½“ę—„ćÆ恻ćØ悓恩ēœ ć‚Œćš。ē—›ćæ恠恑恧čØ€ćˆć°č¦ŖēŸ„ć‚‰ćšęŠœć„ćŸå¾Œć®ę–¹ćŒē—›ć‹ć£ćŸć§ć™。
ć‚¬ćƒ¼ć‚¼ęŠœćć®ēž¬é–“ćÆę°—ęŒć”ć®ę‚Ŗć„ę„Ÿč§¦ć“ćć‚ć‚Œć©、å…Ø恏ē—›ćæćÆć‚ć‚Šć¾ć›ć‚“ć§ć—ćŸ。äø€ēž¬ć ć‘é¼»ć®é€šć‚Šć‚’å®Ÿę„Ÿć§ćć¾ć™ćŒ、ć‚¹ćƒćƒ³ć‚¼ćƒ«ćØē¶æēƒć‚’ć¾ćŸć‚‚ć‚„č©°ć‚č¾¼ć¾ć‚Œć¾ć™。ć“ć®ę—„ć‹ć‚‰ć ć‚“ć ć‚“é£Ÿę¬²ć‚‚ę¹§ć„ć¦ććŸć‚“ć§ć™ćŒ、ć„ć‹ć‚“ć›ć‚“å‘³ćŒę„Ÿć˜ć‚‰ć‚ŒćŖ恄恮恧、恓悌ćÆä»Šć‚“ćƒžć®å‘³ćŒć—ć¦ć„ć‚‹ćÆ恚…ćØč‡Ŗåˆ†ć‚’ę€ć„č¾¼ć¾ć›ćŖćŒć‚‰é£Ÿć¹ć¦ć„ć¾ć—ćŸ。
ęœ€å¾Œć®ć‚¹ćƒćƒ³ć‚¼ćƒ«ć‚’ęŠœć„ćŸå¾ŒćÆę„Ÿå‹•ć™ć‚‹ć»ć©é¼»ćŒé€šć‚‹ć“ćØć‚’ęœŸå¾…ć—ć¦ćŸć‚“ć§ć™ćŒ、å…ƒć€…ē‚¹é¼»č–¬ć‚’ä½æć£ć¦ćŸć“ćØ悂恂悊、ę‰‹č”“å‰ć‚’10恠ćØć—ćŸć‚‰ę‰‹č”“å¾ŒćŒ12ćć‚‰ć„ć®ę„Ÿč¦šć§ć™。ęœę—©ćē—…é™¢ć«č”Œć£ćŸć®ć§ćƒ‰ćƒˆćƒ¼ćƒ«ć§č»½é£Ÿć‚’å–ć‚ć†ćØåÆ„ć£ćŸć‚“ć§ć™ćŒ、é ¼ć‚“ć ćƒ›ćƒƒćƒˆćƒ‰ćƒƒć‚°ć‹ć‚‰ćƒ‘ćƒ³ć®é¦™ć‚ŠćŒę„Ÿć˜ć‚‰ć‚ŒćŸć®ć«ćÆ、ę³£ććć†ć«ćŖć‚Šć¾ć—ćŸšŸŒ­

初čØŗć‹ć‚‰ęœ€å¾Œć®ć‚¹ćƒćƒ³ć‚¼ćƒ«ęŠœćć¾ć§ć®ć‚¹ć‚±ć‚øćƒ„ćƒ¼ćƒ«ćÆ恓悓ćŖꄟ恘恧恗恟。
3/31 初čØŗ CT&č”€ę¶²ę¤œęŸ»
4/9  ę‰‹č”“å‰ć®ē¢ŗčŖ
4/25 ꉋ蔓
4/27 ć‚¬ćƒ¼ć‚¼ęŠœć&ć‚¹ćƒćƒ³ć‚¼ćƒ«č©°ć‚č¾¼ćæ
5/1 ć‚¹ćƒćƒ³ć‚¼ćƒ«ęŠœć

Apr 27, 2018

New EOM Style

仕äŗ‹ć§ä½æć£ć¦ć„ć‚‹ćØć‚ć‚‹č£½å“ćŒēµę§‹ćŖé‡ć®äøå…·åˆćŒć‚ć‚‹ēŠ¶ę…‹ć§、EOM恫ćŖć£ć¦ć—ć¾ć„ę‚²ć—ć„。äøå…·åˆć®ć‚ć‚‹éƒØ分ćÆJavać§å®Ÿč£…ć•ć‚Œć¦ć„ćŸć®ć§、jadć§é€†ć‚³ćƒ³ćƒ‘ć‚¤ćƒ©ć‚’č”Œć„äæ®ę­£ć™ć‚‹ē®‡ę‰€ć®ć‚ćŸć‚Šć‚’ć¤ć‘ć¦ć‹ć‚‰、dirtyJOE恧classćƒ•ć‚”ć‚¤ćƒ«ć‚’ē›“ꎄäæ®ę­£ć—恦ćæ恟悓恧恙恌、ēµęžœēš„恫ćÆē›“悉恚...。é–¢é€£ć™ć‚‹å…Øclassćƒ•ć‚”ć‚¤ćƒ«ć®packagećƒ‘ć‚¹ć‚’å¤‰ę›“ć™ć‚Œć°ē›“悊恝恆ćŖ恓ćØćÆć‚ć‹ć£ćŸć‚“ć§ć™ćŒ、恝悌ćÆć•ć™ćŒć«å¤§å¤‰ć™ćŽć‚‹ć®ć§č«¦ć‚ć¾ć—ćŸ。EOM恫恙悋ćŖć‚‰ć‚³ć‚¢éƒØ分ćÆå…¬é–‹ć—ćŖć„ć«ć›ć‚ˆ、ćƒć‚°ć«é–¢ć‚ć‚‹éƒØåˆ†ć ć‘ć§ć‚‚ćć®ćƒ¦ćƒ¼ć‚¶ćƒ¼ć«å…¬é–‹ć™ć‚‹ć“ćØ恌恧恍悌恰、ćæ悓ćŖćƒćƒƒćƒ”ćƒ¼ć«ćŖć‚Œć‚‹ć®ć§ćÆćØę€ć£ćŸć‚Šć—ć¾ć™。

ę£®åšå—£ć•ć‚“ć®ē™¾å¹“ć‚·ćƒŖćƒ¼ć‚ŗ 1作ē›®ć®「儳ēŽ‹ć®ē™¾å¹“åƆ室」悒čŖ­ćæēµ‚ć‚ć‚Šć¾ć—ćŸ。Wć‚·ćƒŖćƒ¼ć‚ŗ、Xć‚·ćƒŖćƒ¼ć‚ŗ、ē™¾å¹“ć‚·ćƒŖćƒ¼ć‚ŗćØčŖ­ć‚“恧恄恦、å…±é€šć—ć¦é¢ē™½ć„ćØę€ć†ć®ćÆ、äø»äŗŗ公ćØćć®äø»äŗŗå…¬ć‚’ę”Æ恈悋äŗŗ間悂恗恏ćÆć‚¦ć‚©ćƒ¼ć‚«ćƒ­ćƒ³ć®ä¼šč©±ć§ć—ć‚‡ć†ć‹。Wć‚·ćƒŖćƒ¼ć‚ŗćŖć‚‰ćƒć‚®ćƒŖćØ悦悰悤、Xć‚·ćƒŖćƒ¼ć‚ŗćŖć‚‰å°å·ćØēœŸé‹、ē™¾å¹“ć‚·ćƒŖćƒ¼ć‚ŗćŖ悉悵ć‚Ø惐・ćƒŸćƒćƒ«ćØćƒ­ć‚¤ćƒ‡ć‚£。ćƒć‚®ćƒŖćØćƒ‡ćƒœćƒ©ć®ä¼šč©±ć‚‚å„½ćć§ć™ćŒ、å·»ć‚’é‡ć­ć‚‹ć«ć¤ć‚Œć‚¦ć‚°ć‚¤ć®č‰Æ恕恌悈悊際ē«‹ć£ć¦ć„ć‚‹ćØ僕ćÆę„Ÿć˜ć¾ć™。Wć‚·ćƒŖćƒ¼ć‚ŗćÆ恝悍恝悍ēµ‚ć‚ć‚Šć®é›°å›²ę°—ćŒå‡ŗć¦ć„ć¦č‹„å¹²åÆ‚ć—ć„ć§ć™ćŒ、ęœ€å¾Œć¾ć§ę„½ć—ćæ恧恙。

Apr 17, 2018

Kubernetes Up & Running


å…„é–€ Kubernetes悒čŖ­ćæć¾ć—ćŸ。ęœ¬ć®äø­ć§ē“¹ä»‹ć•ć‚Œć¦ć„ć‚‹yamlćÆkubernetes-up-and-running/examples恫恂悋ćæ恟恄恧恙。

# Docker
docker run -d --name kuard --publish 8081:8080 --memory 200m --memory-swap 1G --cpu-shares 1024 gcr.io/kuar-demo/kuard-amd64:1
docker stop kuard; docker rm kuard

# Kubernetes
kubectl version
kubectl get componentstatuses

controller-manager' ć‚Æćƒ©ć‚¹ć‚æäøŠć®ęŒÆć‚‹čˆžć„ć‚’åˆ¶å¾”ć™ć‚‹
scheduler 各Pod悒ć‚Æćƒ©ć‚¹ć‚æå†…ć®ćć‚Œćžć‚Œć®ćƒŽćƒ¼ćƒ‰ć«é…ē½®ć™ć‚‹
etcd ć‚Æćƒ©ć‚¹ć‚æć®ć™ć¹ć¦ć®APIć‚Ŗ惖ć‚ø悧ć‚Æ惈恌äæå­˜ć•ć‚Œć‚‹ć‚¹ćƒˆćƒ¬ćƒ¼ć‚ø

ćƒŽćƒ¼ćƒ‰ć®č”Øē¤ŗ恙悋
kubectl get nodes
masterćƒŽćƒ¼ćƒ‰ APIć‚µćƒ¼ćƒćƒ¼ć‚„ć‚¹ć‚±ć‚øćƒ„ćƒ¼ćƒ©ć®ć‚³ćƒ³ćƒ†ćƒŠćŒå‹•ć
workerćƒŽćƒ¼ćƒ‰ ćƒ¦ćƒ¼ć‚¶ćŒä½œęˆć—ćŸć‚³ćƒ³ćƒ†ćƒŠćŒå‹•ć

ē‰¹å®šćƒŽćƒ¼ćƒ‰ć®č”Øē¤ŗ恙悋
kubectl describe nodes xxx

ćƒ—ćƒ­ć‚­ć‚·ć®äø€č¦§ć‚’č”Øē¤ŗ恙悋
kubectl get daemonSets --namespace=kube-system kube-proxy

DNS恮äø€č¦§ć‚’č”Øē¤ŗ恙悋
kubectl get deployments --namespace=kube-system kube-dns

DNSć‚µćƒ¼ćƒć‚’ćƒ­ćƒ¼ćƒ‰ćƒćƒ©ćƒ³ć‚·ćƒ³ć‚°ć™ć‚‹ćŸć‚ć®Kubernetes Service悒č”Øē¤ŗ恙悋
kubectl get services --namespace=kube-system kube-dns

UIć‚µćƒ¼ćƒć®ē¢ŗčŖ
kubectl get deployments --namespace=kube-system kube-dashboard

ćƒ­ćƒ¼ćƒ‰ćƒćƒ©ćƒ³ć‚¹ć®ćŸć‚ć®Service悒č”Øē¤ŗ
kubectl get services --namespace=kube-system kube-dashboard

UIć«ć‚¢ć‚Æć‚»ć‚¹ć™ć‚‹
kubectl proxy

# 悈恏ä½æ恆kubectlć‚³ćƒžćƒ³ćƒ‰
NamespacećÆć‚Ŗ惖ć‚ø悧ć‚Æćƒˆć®é›†ć¾ć‚Šć‚’å…„ć‚Œć‚‹ćƒ•ć‚©ćƒ«ćƒ€ć®ć‚ˆć†ćŖć‚¤ćƒ”ćƒ¼ć‚ø

å®Ÿč”Œäø­ć®ć‚³ćƒ³ćƒ†ćƒŠć®ćƒ­ć‚°ć‚’ē¢ŗčŖć™ć‚‹
kubectl logs podname
kubectl logs -f podname

ćƒ•ć‚”ć‚¤ćƒ«ć®ć‚„ć‚ŠćØć‚Šć‚’č”Œć†
kubectl cp podname:/path/to/remote/file /path/to/local/file

å®Ÿč”Œäø­ć®ć‚³ćƒ³ćƒ†ćƒŠć«ćƒ­ć‚°ć‚¤ćƒ³ć™ć‚‹
kubectl exec -it podname -- bash

Kubernetes恧ćÆć‚³ćƒ³ćƒ†ćƒŠć§ćÆćŖ恏、PodćŒęœ€å°ć®ćƒ‡ćƒ—ćƒ­ć‚¤å˜ä½ć§ć‚ć‚Š、1恤恮PodćŖć„ć®ć‚³ćƒ³ćƒ†ćƒŠćÆć™ć¹ć¦åŒć˜ćƒžć‚·ćƒ³äøŠć«é…ē½®ć•ć‚Œć‚‹
ć‚¹ć‚±ćƒ¼ćƒ«ć•ć›ć‚‹ę™‚ć‚‚Pod単位ćŖ恮恧、ä¾‹ćˆć°WordPressćØMySQLć‚’åŒć˜Pod恫恙悋ćØē‰‡ę–¹ć ć‘ć‚¹ć‚±ćƒ¼ćƒ«ć•ć›ć‚‹ć“ćØ恌恧恍ćŖ恄
ć‚³ćƒ³ćƒ†ćƒŠćŒé•ć†ćƒžć‚·ćƒ³ć«é…ē½®ć•ć‚Œć¦ć‚‚ę­£åøøć«å‹•ä½œć§ćć‚‹ć®ć§ć‚ć‚Œć°、Podć‚’åˆ†ć‘ć‚‹ć®ćŒę­£č§£ćŖåÆčƒ½ę€§ćŒé«˜ć„

Podć®ä½œęˆ、ē¢ŗčŖ、削除
kubectl run kuard --image=gcr.io/kuar-demo/kuard-amd64:1
kubectl get pods
kubectl delete deployments/kuard

Pod悒yamlć§ä½œęˆć™ć‚‹
apiVersion: v1
kind: Pod
metadata:
  name: kuard
spec:
  containers:
    - image: gcr.io/kuar-demo/kuard-amd64:1
      name: kuard
      ports:
        - containerPort: 8081
          name: http
          protocol: TCP

kubectl apply -f kuard-pod.yaml

Podć®č©³ć—ć„ęƒ…å ±ć®ē¢ŗčŖ
kubectl describe pods kuard

č©³ē“°ć‚’ē¢ŗčŖć™ć‚‹ć‚Ŗćƒ—ć‚·ćƒ§ćƒ³
kubectl get pods -o wide
kubectl get pods -o json
kubectl get pods -o yaml

Podć®å‰Šé™¤
kubectl delete pods/kuard
kubectl delete -f kuard-pod.yaml

Podć®å‰Šé™¤ć‚³ćƒžćƒ³ćƒ‰ć‚’å®Ÿč”Œć—ćŸę™‚ć«、ć¾ćšTerminatingćØ恄恆ēŠ¶ę…‹ć«ćŖć‚Šćƒ‡ćƒ•ć‚©ćƒ«ćƒˆ30ē§’恮ēŒ¶äŗˆęœŸé–“(grace period)å¾Œć«å‰Šé™¤ć•ć‚Œć‚‹。TerminatingēŠ¶ę…‹ć«ćŖ悋ćØćć®PodćÆćƒŖć‚Æć‚Øć‚¹ćƒˆć‚’å—ć‘ä»˜ć‘ćŖ恄。恓恮ēŒ¶äŗˆęœŸé–“ć«ć‚ˆć£ć¦å‡¦ē†äø­ć®åÆčƒ½ę€§ć®ć‚ć‚‹ćƒŖć‚Æć‚Øć‚¹ćƒˆć‚’Podć‚’å‰Šé™¤ć™ć‚‹å‰ć«ēµ‚ć‚ć‚‰ć‚Œć‚‹ć“ćØ恌恧恍悋。

ćƒćƒ¼ćƒˆćƒ•ć‚©ćƒÆćƒ¼ćƒ‰(ć‚³ćƒžćƒ³ćƒ‰å®Ÿč”Œäø­ć®ćæęœ‰åŠ¹)
kubectl port-forward kuard 8081:8080

ćƒ­ć‚°ć®ē¢ŗčŖ
kubectl logs kuard
kubectl logs -f kuard
kubectl logs --previous kuard

ćƒ˜ćƒ«ć‚¹ćƒć‚§ćƒƒć‚Æ
ćƒ—ćƒ­ć‚»ć‚¹ćƒ˜ćƒ«ć‚¹ćƒć‚§ćƒƒć‚Æę©Ÿčƒ½ć«ć‚ˆć£ć¦ć‚¢ćƒ—ćƒŖć‚±ćƒ¼ć‚·ćƒ§ćƒ³ć®ćƒ”ć‚¤ćƒ³ćƒ—ćƒ­ć‚»ć‚¹ćŒå‹•ć„ć¦ć„ć‚‹ć‹åøø恫ē›£č¦–恗、å‹•ć„ć¦ć„ćŖć„å “åˆćÆKubernetesćŒćƒ—ćƒ­ć‚»ć‚¹ć‚’å†čµ·å‹•ć™ć‚‹。ćƒ—ćƒ­ć‚»ć‚¹ćŒē”Ÿćć¦ć„悋悂恮恮、ćƒŖć‚Æć‚Øć‚¹ćƒˆć«ćÆåæœē­”ć§ćć¦ć„ćŖć„å “åˆć«ćÆ恓悌恠恑恧ćÆåƾåæœć§ććŖ恄。

Liveness probe
apiVersion: v1
kind: Pod
metadata:
  name: kuard
spec:
  containers:
    - image: gcr.io/kuar-demo/kuard-amd64:1
      name: kuard
      livenessProbe:
        httpGet:
          path: /healthy
          port: 8080
        initialDelaySeconds: 5
        timeoutSeconds: 1
        periodSeconds: 10
        failureThreshold: 3
      ports:
        - containerPort: 8081
          name: http
          protocol: TCP

initialDelaySecondsćÆPodå†…ć®å…Øć‚³ćƒ³ćƒ†ćƒŠćŒä½œęˆć•ć‚Œć¦ć‹ć‚‰5ē§’ēµŒéŽć™ć‚‹ć¾ć§ćƒŖć‚Æć‚Øć‚¹ćƒˆć‚’é€ć‚‰ćŖ恄
timeoutSeconds恮1ē§’ä»„å†…ć«åæœē­”恙悋åæ…č¦ćŒć‚ć‚‹
HTTP惁悧惃ć‚Æ恠恑恧ćŖ恏tcpSocketć‚‚ć‚µćƒćƒ¼ćƒˆć—ć¦ć„ć‚‹
execē›£č¦–ć‚‚åÆčƒ½ć§、ć‚³ćƒ³ćƒ†ćƒŠå†…ć§ć‚¹ć‚ÆćƒŖćƒ—ćƒˆć®čæ”ć‚Šå€¤ćŒ0ćŖć‚‰ęˆåŠŸ、0仄外ćŖć‚‰å¤±ę•—ć®ć‚ˆć†ćŖ恓ćØ悂恧恍悋
https://kubernetes.io/docs/tasks/configure-pod-container/configure-liveness-readiness-probes/

ćƒŖć‚½ćƒ¼ć‚¹č¦ę±‚ ęœ€ä½Žé™åæ…要ćŖćƒŖć‚½ćƒ¼ć‚¹
ćƒŖć‚½ćƒ¼ć‚¹åˆ¶é™ ęœ€å¤§ćƒŖć‚½ćƒ¼ć‚¹é‡
apiVersion: v1
kind: Pod
metadata:
  name: kuard
spec:
  containers:
    - image: gcr.io/kuar-demo/kuard-amd64:1
      name: kuard
      resources:
        requests:
          cpu: "500m"
          memory: "128Mi"
      ports:
        - containerPort: 8081
          name: http
          protocol: TCP
1CPUć®åŠåˆ†ćŠć‚ˆć³128MB悒ćƒŖć‚½ćƒ¼ć‚¹č¦ę±‚ć«čØ­å®šć™ć‚‹ä¾‹
CPU恮ćƒŖć‚½ćƒ¼ć‚¹č¦ę±‚ćÆLinuxć‚«ćƒ¼ćƒćƒ«ć®cpu-sharesę©Ÿčƒ½ć‚’ä½æć£ć¦å®Ÿč£…ć•ć‚Œć¦ć„ć‚‹

apiVersion: v1
kind: Pod
metadata:
  name: kuard
spec:
  containers:
    - image: gcr.io/kuar-demo/kuard-amd64:1
      name: kuard
      resources:
        requests:
          cpu: "500m"
          memory: "128Mi"
        limits:
          cpu: "1000m"
          memory: "256Mi"         
      ports:
        - containerPort: 8081
          name: http
          protocol: TCP
ćƒŖć‚½ćƒ¼ć‚¹åˆ¶é™ć«1CPU、ćƒ”ćƒ¢ćƒŖ256MB悒čØ­å®šć™ć‚‹ä¾‹

ćƒ‡ćƒ¼ć‚æ恮ę°øē¶šåŒ–
spec.volumnsć‚»ć‚Æć‚·ćƒ§ćƒ³ć‚‚ć—ććÆvolumnMounts悒ä½æē”Ø恙悋
apiVersion: v1
kind: Pod
metadata:
  name: kuard
spec:
  volumes:
    - name: "kuard-data"
      hostPath:
        path: "/var/lib/kuard"
  containers:
    - image: gcr.io/kuar-demo/kuard-amd64:1
      name: kuard
      volumeMounts:
        - mountPath: "/data"
          name: "kuard-data"
      ports:
        - containerPort: 8081
          name: http
          protocol: TCP
kuard-dataćØ恄恆惜ćƒŖćƒ„ćƒ¼ćƒ ć‚’å®šē¾©ć—、kuardć‚³ćƒ³ćƒ†ćƒŠć®/datać«ćƒžć‚¦ćƒ³ćƒˆć™ć‚‹ä¾‹
empDirćÆć‚­ćƒ£ćƒƒć‚·ćƒ„ć«ä¾æ利
NFSć‚µćƒ¼ćƒćƒ¼ć‚’åˆ©ē”Ø恙悋恓ćØ悂åÆčƒ½
  volumes:
    - name: "kuard-data"
      nfs:
        server: my.nfs.server.local
        path: "/exports"


# LabelćØAnnotation
LabelćØAnnotation恮ä½æć„åˆ†ć‘ćÆ、儽ćæć«ć‚‚ć‚ˆć‚‹。ć‚»ćƒ¬ć‚Æć‚æćØ恗恦ä½æ恄恟恏ćŖć£ćŸć‚‰Labelć«ę˜‡ę ¼ć•ć›ć‚‹ć®ćŒč‰Æ恄

kubectl run alpaca-prod --image=gcr.io/kuar-demo/kuard-amd64:1 --replicas=2 --labels="ver=1,app=alpaca,env=prod"
kubectl run alpaca-test --image=gcr.io/kuar-demo/kuard-amd64:2 --replicas=1 --labels="ver=2,app=alpaca,env=test"
kubectl run bandicoot-prod --image=gcr.io/kuar-demo/kuard-amd64:2 --replicas=2 --labels="ver=2,app=bandicoot,env=prod"
kubectl run bandicoot-staging --image=gcr.io/kuar-demo/kuard-amd64:2 --replicas=1 --labels="ver=2,app=bandicoot,env=staging"

Label恮ē¢ŗčŖ
kubectl get deployments --show-labels
kubectl get deployments -L env
kubectl get pods --selector="ver=2"
kubectl get pods --selector="ver!=2"
kubectl get pods --selector="app=bandicoot,ver=2"
kubectl get pods --selector="app in (alpaca,bandicoot)"
kubectl get pods --selector="app notin (alpaca,bandicoot)"
kubectl get deployments --selector="canary"
kubectl get deployments --selector="!canary"

Label恮čæ½åŠ (ReplicaSet悄PodćÆå¤‰ę›“ć•ć‚ŒćŖ恄恮恧ę³Øꄏ)
kubectl label deployments alpaca-test "canary=true"

Deploymentć®å‰Šé™¤
kubectl delete deployments --selector="canary"
kubectl delete deployments --all

# ć‚µćƒ¼ćƒ“ć‚¹ćƒ‡ć‚£ć‚¹ć‚«ćƒćƒŖ
kubectl run alpaca-prod --image=gcr.io/kuar-demo/kuard-amd64:1 --replicas=3 --port=8080 --labels="ver=1,app=alpaca,env=prod"
kubectl expose deployment alpaca-prod
kubectl run bandicoot-prod --image=gcr.io/kuar-demo/kuard-amd64:2 --replicas=2 --port=8080 --labels="ver=2,app=bandicoot,env=prod"


kubectl expose deployment bandicoot-prod
kubectl get services -o wide

ALPACA_POD=$(kubectl get pods -l app=alpaca -o jsonpath='{.items[0].metadata.name}')
kubectl port-forward $ALPACA_POD 48858:8080

Readiness probe
kubectl edit deployment/alpaca-prod

    spec:
      containers:
      - image: gcr.io/kuar-demo/kuard-amd64:1
        imagePullPolicy: IfNotPresent
        name: alpaca-prod
        readinessProbe:
          httpGet:
            path: /ready
            port: 8080
          periodSeconds: 2
          initialDelaySeconds: 0
          failureThreshold: 3
          successThreshold: 1
        ports:
        - containerPort: 8080
          protocol: TCP

Service恮Endpoints恫åÆ¾ć™ć‚‹ćƒˆćƒ©ćƒ•ć‚£ćƒƒć‚Æ恮ē¢ŗčŖ
kubectl get endpoints alpaca-prod --watch

Endpointsć‚Ŗ惖ć‚ø悧ć‚Æ惈恮ē¢ŗčŖ
kubectl describe endpoints alpaca-prod

ꖰ恗恄ServicećŒä½œć‚‰ć‚Œć‚‹ćØkube-proxyćÆ、iptablesć®ćƒ«ćƒ¼ćƒ«ć‚’ćƒ›ć‚¹ćƒˆć®ć‚«ćƒ¼ćƒćƒ«ć«čØ­å®šć™ć‚‹

# ReplicaSet
å†—é•·ę€§、ć‚¹ć‚±ćƒ¼ćƒ«、ć‚·ćƒ£ćƒ¼ćƒ‡ć‚£ćƒ³ć‚°ć®ćŸć‚ć«ē”Øę„ć•ć‚Œć¦ć„ć‚‹ć®ćŒReplicaSet
ReplicaSetćÆPod恮Label悒ä½æć£ć¦ć‚Æćƒ©ć‚¹ć‚æ恮ēŠ¶ę…‹ć‚’ē›£č¦–恙悋
äøå…·åˆć®ć‚ć‚‹Pod恮Labelć ć‘ć‚’å¤‰ę›“ć—、 ReplicaSet恋悉Podć‚’åˆ‡ć‚Šé›¢ć™ćØ、PodćÆčµ·å‹•ć—ćŸć¾ć¾ć«ćŖć‚‹ć®ć§ćƒ‡ćƒćƒƒć‚°ćŒć—ć‚„ć™ććŖ悋

ReplicaSetć®ęœ€å°é™ć®å®šē¾©ä¾‹(kuard-rs.yaml)
apiVersion: extensions/v1beta1
kind: ReplicaSet
metadata:
  name: kuard-rs
spec:
  replicas: 1
  template:
    metadata:
      labels:
        app: kuard
        version: "2"
    spec:
      containers:
        - name: kuard
          image: "gcr.io/kuar-demo/kuard-amd64:2"
kubectl apply -f kuard-rs.yaml

Pod恋悉ReplicaSet恮ē‰¹å®š
kubectl get pods kuard-rs-4m2nj -o yaml | grep owner -A 

ReplicaSet恫åƾåæœć™ć‚‹Podć®é›†åˆć®ē‰¹å®š
kubectl get pods -l app=kuard,version=2

kubectl scale悒ä½æć£ćŸå‘½ä»¤ēš„ć‚¹ć‚±ćƒ¼ćƒ«(悂恗ä½æē”Øć™ć‚‹å “åˆćÆyaml悂ę›ø恍ꏛ恈悋恓ćØ)
kubectl scale replicasets kuard-rs --replicas=4

宣č؀ēš„ć‚¹ć‚±ćƒ¼ćƒ«
yaml恮replicasć®å€¤ć‚’ę›øćę›ćˆå¾Œć«kubectl apply -f kuard-rs.yaml

ę°“å¹³Podć‚Ŗćƒ¼ćƒˆć‚¹ć‚±ćƒ¼ćƒŖćƒ³ć‚°(HPA Horizontal Pod Autoscaling)
heapsterćØ恄恆Podć«ć‚ˆć‚Šćƒ”ćƒˆćƒŖć‚Æć‚¹ć‚’čæ½č·”恗、HPAćŒć‚¹ć‚±ćƒ¼ćƒŖćƒ³ć‚°ć®åˆ¤ę–­ć‚’č”Œć†ę™‚ć«ä½æē”Øć™ć‚‹ćƒ”ćƒˆćƒŖć‚Æć‚¹ć‚’å–å¾—ć™ć‚‹APIć‚’ęä¾›ć™ć‚‹
ć‚Æćƒ©ć‚¹ć‚æå†…ć«heapsterćŒå­˜åœØ恗ćŖć„å “åˆćÆHPAćÆę­£åøøć«å‹•ä½œć—ćŖ恄
Podć®ćƒ¬ćƒ—ćƒŖć‚«ć‚’čæ½åŠ ć™ć‚‹ć“ćØć‚’ę°“å¹³ć‚¹ć‚±ćƒ¼ćƒ«、Pod恫åæ…要ćŖćƒŖć‚½ćƒ¼ć‚¹ć‚’å¢—ć‚„ć™ć“ćØć‚’åž‚ē›“ć‚¹ć‚±ćƒ¼ćƒ«ćØåŒŗåˆ„ć—ć¦ć„ć‚‹

CPUä½æē”ØēŽ‡ć‚’å…ƒć«ć—ćŸć‚Ŗćƒ¼ćƒˆć‚¹ć‚±ćƒ¼ćƒ«
kubectl autoscale rs kuard-rs --min=2 --max=5 --cpu-percent=80
CPUä½æē”ØēŽ‡80%ć‚’é–¾å€¤ć«ć—ć¦、ćƒ¬ćƒ—ćƒŖć‚«ę•°2〜5ć®é–“ć§ć‚¹ć‚±ćƒ¼ćƒ«ć™ć‚‹ć‚Ŗćƒ¼ćƒˆć‚¹ć‚±ćƒ¼ćƒ©

ē¢ŗčŖ
kubectl get hpa

ReplicaSetć‚’å‰Šé™¤ć™ć‚‹ćØē®”ē†ć—恦恄悋Pod悂äø€ē·’ć«å‰Šé™¤ć•ć‚Œć‚‹
kubectl delete rs kuard-rs

ReplicaSet恮ćæå‰Šé™¤ć—ćŸć„å “åˆ
kubectl delete rs kuard-rs --cascade=false

# Daemon Set
ReplicaSetćØ違恄、ćƒŽćƒ¼ćƒ‰ć‚»ćƒ¬ć‚Æć‚æ悒ä½æ悏ćŖ恄限悊、å…ØćƒŽćƒ¼ćƒ‰ć«Podć‚’ä½œć‚‹

fluentd.yaml
apiVersion: extensions/v1beta1
kind: DaemonSet
metadata:
  name: fluentd
  namespace: kube-system
  labels:
    app: fluentd
spec:
  template:
    metadata:
      labels:
        app: fluentd
    spec:
      containers:
      - name: fluentd
        image: fluent/fluentd:v0.14.10
        resources:
          limits:
            memory: 200Mi
          requests:
            cpu: 100m
            memory: 200Mi
        volumeMounts:
        - name: varlog
          mountPath: /var/log
        - name: varlibdockercontainers
          mountPath: /var/lib/docker/containers
          readOnly: true
      terminationGracePeriodSeconds: 30
      volumes:
      - name: varlog
        hostPath:
          path: /var/log
      - name: varlibdockercontainers
        hostPath:
          path: /var/lib/docker/containers
kubectl apply -f fluentd.yaml

Node恫Label悒čæ½åŠ ć—、ē¢ŗčŖ
kubectl label nodes minikube ssd=true
kubectl get nodes --show-lables
kubectl get nodes --selector ssd

nginx-fast-storage.yaml(ssd=truećØ恄恆LabelćŒćƒŽćƒ¼ćƒ‰ć§ć®ćæå‹•ćDaemonSet)
apiVersion: extensions/v1beta1
kind: "DaemonSet"
metadata:
  labels:
    app: nginx
    ssd: "true"
  name: nginx-fast-storage
spec:
  template:
    metadata:
      labels:
        app: nginx
        ssd: "true"
    spec:
      nodeSelector:
        ssd: "true"
      containers:
        - name: nginx
          image: nginx:1.10.0
ä»–ć®ćƒŽćƒ¼ćƒ‰ć«ssd=truećØ恄恆Label悒čæ½åŠ ć™ć‚Œć°、ćć®ćƒŽćƒ¼ćƒ‰ć«ć‚‚ćƒ‡ćƒ—ćƒ­ć‚¤ć•ć‚Œć‚‹。逆恫Labelć‚’å‰Šé™¤ć™ć‚‹ćØćć®PodćÆDaemonSetć‚³ćƒ³ćƒˆćƒ­ćƒ¼ćƒ©ć‹ć‚‰å‰Šé™¤ć•ć‚Œć‚‹

forćƒ«ćƒ¼ćƒ—ć‚’ē”Ø恄恟Podć®å‰Šé™¤
PODS=$(kubectl get pods -o jsonpath -template='{.ites[*].metadata.name}')
for x in $PODS; do
  kubectl delete pods ${x}
  sleep 60
done

ćƒ­ćƒ¼ćƒŖćƒ³ć‚°ć‚¢ćƒƒćƒ—ćƒ‡ćƒ¼ćƒˆć‚’ä½æ恆恫ćÆspec.updateStrategy.typećƒ•ć‚£ćƒ¼ćƒ«ćƒ‰ć«RollingUpdate悒čØ­å®šć™ć‚‹。ćƒ­ćƒ¼ćƒŖćƒ³ć‚°ć‚¢ćƒƒćƒ—ćƒ‡ćƒ¼ćƒˆäø­ć®ć‚¹ćƒ†ćƒ¼ć‚æć‚¹ć®ē¢ŗčŖćÆkubectl rolloutć‚³ćƒžćƒ³ćƒ‰ć‚’ä½æē”Ø恙悋
kubectl rollout status daemonSets nginx-fast-storage

DaemonSetć®å‰Šé™¤
kubectl delete -f fluentd.yaml

# Job
1å›žé™ć‚Šć®å‡¦ē†ć‚’č”Œć†ć®ć«ä¾æ利ćŖć®ćŒJobć‚Ŗ惖ć‚ø悧ć‚Æ惈
kubectl run -i oneshot --image=gcr.io/kuar-demo/kuard-amd64:1 --restart=OnFailure -- --keygen-enable --keygen-exit-on-complete --keygen-num-to-gen 10
-ić‚Ŗćƒ—ć‚·ćƒ§ćƒ³ćÆć‚³ćƒžćƒ³ćƒ‰ćŒåÆ¾č©±ćƒ¢ćƒ¼ćƒ‰ć§ć‚ć‚‹ć“ćØ悒č”Ø恙
--restart=OnFailurećÆkubectl恫Jobć‚Ŗ惖ć‚ø悧ć‚Æćƒˆć‚’ä½œęˆć™ć‚‹ć‚ˆć†ęŒ‡ē¤ŗ恙悋

完äŗ†å¾Œć®Podć‚‚å«ć‚ć¦č”Øē¤ŗ恙悋
kubectl get pods -a

yaml恧Jobć‚’ä½œęˆć™ć‚‹ä¾‹
kubectl apply -f job-oneshot.yaml

å¤±ę•—ć™ć‚‹Jobć®ä¾‹
kubectl apply -f job-oneshot-failure1.yaml
kubectl get pods -a -l job-name=oneshot
KubernetesćÆ恓恮Pod恌CrashLoopBackOff恠ćØčŖč­˜ć—恦恄悋。ć‚Æćƒ©ćƒƒć‚·ćƒ„ć®ē¹°ć‚Ščæ”ć—ć«ć‚ˆć£ć¦ćƒŽćƒ¼ćƒ‰ć®ćƒŖć‚½ćƒ¼ć‚¹ćŒä½æ悏悌過恎ćŖć„ć‚ˆć†ć«、Podć®å†čµ·å‹•ć¾ć§č‡Ŗå‹•ć§å°‘ć—å¾…ć¤。
restartPolicy: Never恫čØ­å®šć™ć‚‹ćØkubeletćÆJobå¤±ę•—ę™‚ć«Podć‚’å†čµ·å‹•ć›ćš、ä»£ć‚ć‚Šć«Podć‚’å¤±ę•—ćØ宣čØ€ć—、ä»£ć‚ć‚Šć®Podć‚’ä½œęˆć™ć‚‹。restartPolicy: OnFailureć«ć—ć¦ćŠćć“ćØ悒ęŽØå„Ø。

äø€å®šę•°ęˆåŠŸć™ć‚‹ć¾ć§äø¦åˆ—å®Ÿč”Œ
kubectl apply -f job-parallel.yaml
kubectl get pods -w
--watchćƒ•ćƒ©ć‚°ć«ć‚ˆć£ć¦ę™‚é–“ćØćØ悂恫Pod恮äø€č¦§ćŒę›“ꖰ恕悌悋恓ćØ悒ē¢ŗčŖć§ćć‚‹

äø¦åˆ—å®Ÿč”Œć‚­ćƒ„ćƒ¼ć‚’å®Ÿč£…ć™ć‚‹ä¾‹
1. 1å›žé™ć‚Šć®ć‚­ćƒ„ćƒ¼ćƒ‡ćƒ¼ćƒ¢ćƒ³ć‚’ē®”ē†ć™ć‚‹ćŸć‚ć«、ReplicaSetć‚’ä½œęˆć™ć‚‹
kubectl apply -f rs-queue.yaml

2. ć‚­ćƒ„ćƒ¼ć«ęŽ„ē¶šć™ć‚‹ćŸć‚ć«ćƒćƒ¼ćƒˆćƒ•ć‚©ćƒÆćƒ¼ćƒ‰ć‚’čØ­å®šć™ć‚‹
QUEUE_POD=$(kubectl get pods -l app=work-queue,component=queue -o jsonpath='{.items[0].metadata.name}')
kubectl port-forward $QUEUE_POD 8080:8080
MemQ Serverć‚’é–‹ć„ć¦ć‚­ćƒ„ćƒ¼ćŒå‹•ä½œć—ć¦ć„ć‚Œć°、ć‚­ćƒ„ćƒ¼ć®ćƒ—ćƒ­ćƒ‡ćƒ„ćƒ¼ć‚µćØć‚³ćƒ³ć‚·ćƒ„ćƒ¼ćƒžćŒDNSēµŒē”±ć§ć‚­ćƒ„ćƒ¼ć‚’ä½æ恈悋ēŠ¶ę…‹。

3. ć‚­ćƒ„ćƒ¼ć®Serviceć‚’ä½œęˆć™ć‚‹
kubectl apply -f service-queue.yaml

4. ć‚µćƒ–ć‚æć‚¹ć‚Æć‚’ć‚­ćƒ„ćƒ¼ć«å…„ć‚Œ、ēŠ¶ę…‹ć‚’ē¢ŗčŖ
for in in work-item-{0..99}; do
  curl -X POST localhost:8080/memq/server/queues/keygen/enqueue -d "$i"
done
curl 127.0.0.1:8080/memq/server/stats

5. ć‚³ćƒ³ć‚·ćƒ„ćƒ¼ćƒžJobć‚’ä½œęˆć™ć‚‹
kubectl apply -f job-consumers.yaml
kubectl get pods恧5恤恮Pod恌äø¦åˆ—ć«å‹•ä½œć—ć¦ć„ć‚‹ć“ćØ恌悏恋悋

# ConfigMapćØSecret
ConfigMap恫ćÆćƒÆćƒ¼ć‚Æćƒ­ćƒ¼ćƒ‰ć«åæœć˜ćŸčØ­å®šęƒ…å ±ć‚’äæå­˜ć™ć‚‹
Secret恫ćÆćƒ‘ć‚¹ćƒÆćƒ¼ćƒ‰ć‚„TLSčØ¼ę˜Žę›øćŖć©ć®ę©ŸåÆ†ęƒ…å ±ć‚’äæå­˜ć™ć‚‹

ConfigMapć®ä½œęˆ
my-config.txt
parameter1 = value1
parameter2 = value2

kubectl create configmap my-config --from-file=my-config.txt --from-literal=extra-param=extra-value --from-literal=another-param=anather-value

kubectl get configmaps my-config -o yaml

ConfigMap恮ä½æē”Ø
kuard-config.yaml
apiVersion: v1
kind: Pod
metadata:
  name: kuard-config
spec:
  containers:
    - name: test-container
      image: gcr.io/kuar-demo/kuard-amd64:1
      imagePullPolicy: Always
      command:
        - "/kuard"
        - "$(EXTRA_PARAM)"
      env:
        - name: ANOTHER_PARAM
          valueFrom:
            configMapKeyRef:
              name: my-config
              key: another-param
        - name: EXTRA_PARAM
          valueFrom:
            configMapKeyRef:
              name: my-config
              key: extra-param
      volumeMounts:
        - name: config-volume
          mountPath: /config
  volumes:
    - name: config-volume
      configMap:
        name: my-config
  restartPolicy: Never
kubectl apply -f kuard-config.yaml
kubectl port-forward kuard-config 8080

Secretć®ä½œęˆ
ć‚µćƒ³ćƒ—ćƒ«ć®TLSć‚­ćƒ¼ćØčØ¼ę˜Žę›øć‚’ćƒ€ć‚¦ćƒ³ćƒ­ćƒ¼ćƒ‰ć™ć‚‹(ć“ć®ä¾‹ä»„å¤–ć§ćÆä½æē”Ø恗ćŖ恄恓ćØ)
curl -o kuard.crt https://storage.googleapis.com/kuar-demo/kuard.crt
curl -o kuard.key https://storage.googleapis.com/kuar-demo/kuard.key
kubectl create secret generic kuard-tls --from-file=kuard.crt --from-file=kuard.key
kubectl describe secrets kuard-tls

Secret恮ä½æē”Ø
ä»Šå›žć®ä¾‹ć§ćÆSecret volume悒/tlsć«ćƒžć‚¦ćƒ³ćƒˆć™ć‚‹ćØ/tls/kuard.crtćŠć‚ˆć³tls/kuard.keyć«ć‚¢ć‚Æć‚»ć‚¹åÆčƒ½ć«ćŖ悋
kuard-secret.yaml
apiVersion: v1
kind: Pod
metadata:
  name: kuard-tls
spec:
  containers:
    - name: kuard-tls
      image: gcr.io/kuar-demo/kuard-amd64:1
      imagePullPolicy: Always
      volumeMounts:
      - name: tls-certs
        mountPath: "/tls"
        readOnly: true
  volumes:
    - name: tls-certs
      secret:
        secretName: kuard-tls
kubectl apply -f kuard-secret.yaml
kubectl port-forward kuard-tls 8443:8443

ćƒ—ćƒ©ć‚¤ćƒ™ćƒ¼ćƒˆDocker惬ć‚øć‚¹ćƒˆćƒŖ
kubectl create secret docker-registry my-image-pull-secret \
 --docker-username=xxx \
 --docker-password=xxx \
 --docker-email=xxx
yamlć®ć‚µćƒ³ćƒ—ćƒ« kuard-secret-ips.yaml

ConfigMapćØSecret恮ē®”ē†
kubectl get secrets
kubectl get configmaps
kubectl describe configmap my-config

# Deployment
Deployment悒ä½æ恆ćØćƒ€ć‚¦ćƒ³ć‚æ悤惠悄ć‚Øćƒ©ćƒ¼ć‚’ē™ŗē”Ÿć•ć›ćšć«ę–°ć—ć„ćƒćƒ¼ć‚øćƒ§ćƒ³ć®ć‚½ćƒ•ćƒˆć‚¦ć‚§ć‚¢ć‚’ć‚·ćƒ³ćƒ—ćƒ«ć«ćƒ­ćƒ¼ćƒ«ć‚¢ć‚¦ćƒˆć§ćć‚‹。
kubectl scale deployments nginx --replicas=2
Deploymentć‚’ć‚¹ć‚±ćƒ¼ćƒ«ć™ć‚‹ćØ、ćć®ē®”ē†äø‹ć«ć‚ć‚‹ReplicaSetć‚‚ć‚¹ć‚±ćƒ¼ćƒ«ć•ć‚Œć‚‹

kubectl get deployments nginx --export -o yaml > nginx-deployment.yaml
kubectl replace -f nginx-deployment.yaml --save-config
kubectl describe deployments nginx
ćƒ­ćƒ¼ćƒ«ć‚¢ć‚¦ćƒˆäø­ćÆOldReplicaSetsćƒ•ć‚£ćƒ¼ćƒ«ćƒ‰ć«å€¤ćŒå…„ć‚‹

ć‚³ćƒžćƒ³ćƒ‰ć§ćÆćŖ恏yamlć§ć‚¹ć‚±ćƒ¼ćƒ«ć•ć›ć‚‹ć«ćÆ仄äø‹ć®éƒØåˆ†ć‚’ę›“ę–°ć™ć‚‹
spec:
  replicas: 3

ćƒ­ćƒ¼ćƒ«ć‚¢ć‚¦ćƒˆć‚’äø­ę–­・å†é–‹ć™ć‚‹
kubectl rollout pause deployments nginx
kubectl rollout resume deployments nginx

ćƒ­ćƒ¼ćƒ«ć‚¢ć‚¦ćƒˆć®å±„ę­“ć‚’ē¢ŗčŖć™ć‚‹
kubectl rollout history deployments nginx
kubectl rollout history deployments nginx --revision=2

ćƒ­ćƒ¼ćƒ«ć‚¢ć‚¦ćƒˆć®ćƒ­ćƒ¼ćƒ«ćƒćƒƒć‚Æ
kubectl rollout undo deployments nginx

å±„ę­“ć®ę•°ć‚’åˆ¶é™ć™ć‚‹ę–¹ę³•
spec:
  revisionHistoryLimit: 14

Deploymentꈦē•„
Recreate: Deployment恫åŸŗć„ć„ćŸPod悒å…Øć¦åœę­¢ć™ć‚‹ć“ćØ恧ReplicaSetć«ę–°ć—ć„ć‚¤ćƒ”ćƒ¼ć‚øć‚’ä½œęˆć•ć›ć‚‹。ć‚·ćƒ³ćƒ—ćƒ«ć§é«˜é€Ÿć ćŒéšœå®³ć‚’čµ·ć“ć™åÆčƒ½ę€§ćŒć‚ć‚Š、ćƒ€ć‚¦ćƒ³ć‚æ悤惠悂éæ恑悉悌ćŖ恄。
Rolling Update: 恂悋äø€å®šć®ęœŸé–“、ę–°ę—§ćƒćƒ¼ć‚øćƒ§ćƒ³ć®äø”ę–¹ćŒćƒŖć‚Æć‚Øć‚¹ćƒˆć‚’å—ć‘ćƒˆćƒ©ćƒ•ć‚£ćƒƒć‚Æć‚’å‡¦ē†ć™ć‚‹。ć‚Æćƒ©ć‚¤ć‚¢ćƒ³ćƒˆå“ćŒę–°ę—§ć©ć”ć‚‰ćØć‚‚é€£ęŗ恧恍悋恓ćØćŒé‡č¦ć«ćŖć£ć¦ćć‚‹。

maxUnavailablećƒ‘ćƒ©ćƒ”ćƒ¼ć‚æćÆćƒ­ćƒ¼ćƒŖćƒ³ć‚°ć‚¢ćƒƒćƒ—ćƒ‡ćƒ¼ćƒˆäø­ć«ä½æē”ØäøåÆčƒ½ć«ćŖć£ć¦ć‚‚ć„ć„Podć®ęœ€å¤§ę•°ć‚’ęŒ‡å®šć™ć‚‹。ēµ¶åÆ¾å€¤ć‚‚ć—ććÆćƒ‘ćƒ¼ć‚»ćƒ³ćƒ†ćƒ¼ć‚øć§ęŒ‡å®šć™ć‚‹。ć“ć®å€¤ćÆćƒ­ćƒ¼ćƒŖćƒ³ć‚°ć‚¢ćƒƒćƒ—ćƒ‡ćƒ¼ćƒˆć‚’é€²ć‚ć‚‹é€Ÿåŗ¦ć«ć‚‚é–¢ć‚ć£ć¦ćć‚‹。

maxSurgećƒ‘ćƒ©ćƒ”ćƒ¼ć‚æćÆćƒ­ćƒ¼ćƒ«ć‚¢ć‚¦ćƒˆę™‚ć«ć©ć®ćć‚‰ć„ć®čæ½åŠ ćƒŖć‚½ćƒ¼ć‚¹ć‚’ä½œć‚Œć‚‹ć‹ć‚’åˆ¶å¾”ć™ć‚‹。maxUnavailable悒0、maxSurge悒20%恫čØ­å®šć™ć‚‹ćØ、ćƒ¬ćƒ—ćƒŖć‚«ę•°ć‚’1.2å€ć«å¢—ć‚„ć—ć¦ć‹ć‚‰ćƒ¬ćƒ—ćƒŖć‚«ć®å‰²åˆć‚’ę—§0.8 ꖰ0.2ćøꛓꖰ恙悋。maxSurge悒100%恫čØ­å®šć™ć‚‹ćØćƒ–ćƒ«ćƒ¼ć‚°ćƒŖćƒ¼ćƒ³ćƒ‡ćƒ—ćƒ­ć‚¤ćƒ”ćƒ³ćƒˆć‚’č”Œćˆć‚‹。

minReadySeconds悒ä½æē”Ø恙悋ćØPodćŒčµ·å‹•ć—ć¦ć‹ć‚‰ęŒ‡å®šć—ćŸę™‚é–“å¾Œć«ę¬”ć®ć‚¢ćƒƒćƒ—ćƒ‡ćƒ¼ćƒˆć«ē§»ć‚‹
spec:
  minReadySeconds: 60

ć‚æć‚¤ćƒ ć‚¢ć‚¦ćƒˆć‚’čØ­å®šć™ć‚‹ć«ćÆprogressDeadlineSeconds悒ä½æē”Ø恙悋
spec:
  progressDeadlineSeconds: 600

# ć‚¹ćƒˆćƒ¬ćƒ¼ć‚øć‚½ćƒŖćƒ„ćƒ¼ć‚·ćƒ§ćƒ³ćØKubernetes恮ēµ±åˆ
ć‚»ćƒ¬ć‚Æć‚æ恮ćŖ恄Service
dns-service.yaml
kind: Service
apiVersion: v1
metadata:
  name: external-database
spec:
  type: ExternalName
  externalName: "database.company.com
ExternalNamećØ恄恆ć‚æć‚¤ćƒ—ć®Serviceć‚’ä½œęˆć™ć‚‹ćØ、外éƒØ恮DNSåć‚’ē¤ŗ恙Aćƒ¬ć‚³ćƒ¼ćƒ‰ć®ä»£ć‚ć‚Šć«CNAMEćƒ¬ć‚³ćƒ¼ćƒ‰ć‚’ä½œęˆć™ć‚‹。ć‚Æćƒ©ć‚¹ć‚æå†…ć®ć‚¢ćƒ—ćƒŖć‚±ćƒ¼ć‚·ćƒ§ćƒ³ćŒexternal-database.default.svc.cluster.localćØć„ć†ćƒ›ć‚¹ćƒˆåć‚’DNSåå‰č§£ę±ŗ恙悋ćØ、DNSćƒ—ćƒ­ćƒˆć‚³ćƒ«ćŒdatabase.company.comć«åå‰ć‚’ć‚Ø悤ćƒŖć‚¢ć‚¹ć™ć‚‹。

IPć‚¢ćƒ‰ćƒ¬ć‚¹ć®ćæęä¾›ć•ć‚Œć¦ć„ć‚‹å “åˆćÆć‚µćƒ¼ćƒ“ć‚¹ćØć‚Øćƒ³ćƒ‰ćƒć‚¤ćƒ³ćƒˆć‚’ä½œęˆć™ć‚‹åæ…č¦ćŒć‚ć‚‹
external-ip-service.yaml
kind: Service
apiVersion: v1
metadata:
  name: external-ip-database

external-ip-endpoints.yaml
kind: Endpoints
apiVersion: v1
metadata:
  name: external-ip-database
subsets:
  - addresses:
    - ip: 192.168.0.1
    ports:
    - port: 3306

Persistent Volumnć®ä¾‹
nfs-volume.yaml
apiVersion: v1
kind: PersistentVolume
metadata:
  name: database
  labels:
    volume: my-volume
spec:
  capacity:
    storage: 1Gi
  nfs:
    server: 192.168.0.1
    path: "/exports"

PersistentVolumeClaimć‚Ŗ惖ć‚ø悧ć‚Æ惈悒ä½æć£ć¦PersistentVolumn悒Podć‹ć‚‰å–å¾—ć™ć‚‹
nfs-volume-claim.yaml
kind: PersistentVolumeClaim
apiVersion: v1
metadata:
  name: database
spec:
  resources:
    requests:
      storage: 1Gi
  selector:
    matchLabels:
      volume: my-volume

StatefulSetćÆReplicaSetć«ä¼¼ćŸč¤‡č£½ć•ć‚ŒćŸPodć®ć‚°ćƒ«ćƒ¼ćƒ—。å‰Šé™¤ę™‚ć‚„ć‚¹ć‚±ćƒ¼ćƒ«ćƒ€ć‚¦ćƒ³ę™‚ć«å„ćƒ¬ćƒ—ćƒŖć‚«ćÆć‚¤ćƒ³ćƒ‡ćƒƒć‚Æć‚¹ć®ę•°å­—ćŒå¤§ćć„é †ć‹ć‚‰å‰Šé™¤ć•ć‚Œć‚‹。
kubectl apply -f mongo-simple.yaml

StatefulSetä½œęˆå¾ŒćÆDNSć‚Øćƒ³ćƒˆćƒŖ悒ē®”ē†ć™ć‚‹ćƒ˜ćƒƒćƒ‰ćƒ¬ć‚¹ćŖServiceć‚‚ä½œć‚‹åæ…č¦ćŒć‚ć‚‹。
kubectl apply -f mongo-service.yaml
kubectl exec mongo-0 bash ping mongo-1.mongo

# 実ē”Øēš„ćŖć‚¢ćƒ—ćƒŖć‚±ćƒ¼ć‚·ćƒ§ćƒ³ć®ćƒ‡ćƒ—ćƒ­ć‚¤
Parse, Ghost, Redisć®ćƒ‡ćƒ—ćƒ­ć‚¤ę–¹ę³•ć®ć‚µćƒ³ćƒ—ćƒ«