sampling data

SELECT
 DATE_FORMAT( FROM_UNIXTIME(UNIX_TIMESTAMP(time) - UNIX_TIMESTAMP(time)%(15*60) )
 , '%H:%i %s') AS S_TIME
 , time
 from a;
 -----------------------------------------------------------------------------------------------

 SELECT
 DATE_FORMAT( FROM_UNIXTIME(UNIX_TIMESTAMP(time) - UNIX_TIMESTAMP(time)%(1*60) )
 , '%H:%i %s') AS S_TIME
 , min(val)
 , max(val)
 , avg(val)
 , count(*)
 from a
 group by s_time
 ;


SELECT
 DATE_FORMAT( FROM_UNIXTIME(UNIX_TIMESTAMP(time) - UNIX_TIMESTAMP(time)%(3) )
 , '%H:%i %s') AS S_TIME
 , time
 from a;


 ---------------------------------------------------------------------------------
 mysql> SELECT
    ->  DATE_FORMAT( FROM_UNIXTIME(UNIX_TIMESTAMP(time) - UNIX_TIMESTAMP(time)%(3) )
    ->  , '%H:%i %s') AS S_TIME
    ->  , time
    ->  from a;
+----------+----------------+
| S_TIME   | time           |
+----------+----------------+
| 15:00 00 | 20140318150000 |
| 15:00 00 | 20140318150001 |
| 15:00 00 | 20140318150002 |
| 15:00 03 | 20140318150003 |
| 15:00 03 | 20140318150004 |
| 15:00 03 | 20140318150005 |
| 15:00 06 | 20140318150006 |
| 15:00 06 | 20140318150007 |
| 15:00 06 | 20140318150008 |
| 15:00 09 | 20140318150009 |
| 15:00 09 | 20140318150010 |
| 16:00 00 | 20140318160001 |
| 16:00 00 | 20140318160002 |
| 16:30 00 | 20140318163001 |
| 16:40 00 | 20140318164001 |
| 16:50 00 | 20140318165001 |
| 16:51 00 | 20140318165101 |
| 16:52 00 | 20140318165201 |
| 16:53 00 | 20140318165301 |
| 16:54 00 | 20140318165401 |
| 16:55 00 | 20140318165501 |
| 16:56 00 | 20140318165601 |
| 16:57 00 | 20140318165701 |
| 16:58 00 | 20140318165801 |
| 16:59 00 | 20140318165901 |
| 17:00 00 | 20140318170000 |
| 16:45 00 | 20140318164500 |
| 16:44 00 | 20140318164400 |
+----------+----------------+
28 rows in set (0.00 sec)

android linux libGL issue

I have android SDK installed into ~/android-sdk-linux_x86, so I did

ln -s /usr/lib/libGL.so.1 ~/android-sdk-linux_x86/tools/lib/libGL.so

This solves errors just like linking to /usr/lib does but doesn't require root and doesn't mess with core system directories.

http://stackoverflow.com/questions/11332817/failed-to-load-libgl-so-in-android

linux cut command

cut명령어# 텍스트파일내용중 부분을 지정하여 출력하는 명령어

-. 옵션
-b : byte,특정 자리만 출력함
-d : 인자로 받은 문자열을 기준으로 필드를 구분함
-f : -d옵션과 함께 사용되며, 구분된 필드를 출력함


-b옵션 사용방법

N
N자리
-b1

N,M,O
N자리,M자리,O자리
-b1,3,5

N-
N자리부터 마지막까지
-b10-

N-M
N자리부터 M자리까지
-b5-10

-M
처음부터 M자리까지
-b-10

ex)
[root@os1 /work/edu-cut]# cat test
abcdefghijklmnopqrstuvwxyz
12345678910
[root@os1 /work/edu-cut]# cut test -b5
e
5
[root@os1 /work/edu-cut]# cut test -b5,7,9
egi
579
[root@os1 /work/edu-cut]# cut test -b5-
efghijklmnopqrstuvwxyz
5678910
[root@os1 /work/edu-cut]# cut test -b5-10
efghij
567891
[root@os1 /work/edu-cut]# cut test -b-10
abcdefghij
1234567891
예를 보면 알수 있듯이 행수는 중요하지 않다, 각행의 자리수만을 지정하여 출력한다.

-. /etc/passwd파일에서 필요한 부분만 출력하기
cut명령어의 대상으로 가장 많이 사용되는 파일이 사용자관리 파일인 passwd,group,shadow파일일듯싶다. 이번엔 이 파일들을 필드로 구분하여 필요한 정보만을 출력하는 예를 보겠다.
>/etc/passwd파일을 콜론(:)으로 구분하였을경우의 각 항목들
계정명:Password:UID:GID:설명자:홈디렉토리:사용쉘:youngjae:x:500:500:Youngjae.hwang:/home/youngjae:/bin/bash

계정명과 홈디렉토리만 출력
[root@os1 /work/edu-cut]# cut -d : -f 1,6 /etc/passwd
root:/root
bin:/bin
daemon:/sbin
adm:/var/adm

shell script

#!/bin/sh

CSV_SRC=/data/temp/trace_bistel_data/temp/002
UPLOAD_DFS_PATH=/user/hadoop/test1
TABLE_IMPALA=test1.e1

#----------- end config

cd $CSV_SRC
# create patition
for part in `ls | cut -f1 -d'_' | uniq`
do
    /usr/bin/impala-shell -q 'ALTER TABLE '$TABLE_IMPALA' ADD if not exists PARTITION (day="'${part}'");refresh '$TABLE_IMPALA';'
    echo "${part} - partition created"
done

# import
for file in `ls -a *.*`
do
    YMD=${file:0:8}
    /usr/bin/hdfs dfs -copyFromLocal `pwd`/$file $UPLOAD_DFS_PATH
    /usr/bin/impala-shell -q 'LOAD DATA INPATH "'$UPLOAD_DFS_PATH'/'${file}'" INTO TABLE '$TABLE_IMPALA' PARTITION (DAY="'${YMD}'");'
    echo ${file}_imported_loaded
done
impala-shell -q 'refresh '$TABLE_IMPALA';compute stats '$TABLE_IMPALA';'

var permission

Check the ownership and permissions on /var/empty and /var/empty/sshd.

They must be owned by root (chown root /var/empty; chown root /var/empty/sshd) and not be writable by anyone except root.

chmod go-w /var/empty/sshd;

chmod go-w /var/empty