Posted Updated  HS 2 hours read (About 14574 words)

10분 완성 Pandas

Pandas 10분 완성 필사

원본: https://pandas.pydata.org/pandas-docs/stable/user_guide/10min.html

번역본: https://dataitgirls2.github.io/10minutes2pandas/

1
2
3
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt

Object Creation (객체 생성)

  • Pandas는 값을 가지고 있는 리스트를 통해 Series를 만들고, 정수로 만들어진 인덱스를 기본값으로 불러올 것입니다.
1
2
s = pd.Series([1,3,5,np.nan,6,8])
s
0    1.0
1    3.0
2    5.0
3    NaN
4    6.0
5    8.0
dtype: float64
  • datetime 인덱스와 레이블이 있는 열을 가지고 있는 numpy 배열을 전달하여 데이터프레임을 만듭니다.
1
2
dates = pd.date_range('20130101', periods=6)
dates
DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
               '2013-01-05', '2013-01-06'],
              dtype='datetime64[ns]', freq='D')

1
2
df = pd.DataFrame(np.random.randn(6,4), index=dates, columns=list('ABCD'))
df

A B C D
2013-01-01 0.300728 -0.263258 0.231729 -0.586384
2013-01-02 -1.099834 -1.311782 1.250473 -0.149189
2013-01-03 -0.348645 -0.913132 0.087372 -0.643829
2013-01-04 -0.591139 1.840492 1.067977 1.738770
2013-01-05 -0.157689 -0.352707 -0.331992 0.634488
2013-01-06 -1.640009 -0.620559 -1.613156 -2.163666 

  <script>
    const buttonEl =
      document.querySelector('#df-967c470c-7432-4134-a124-707e64036844 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-967c470c-7432-4134-a124-707e64036844');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • Series와 같은 것으로 변환될 수 있는 객체들의 dict로 구성된 데이터프레임을 만듭니다.
1
2
3
4
5
6
7
df2 = pd.DataFrame({'A' : 1.,
                    'B' : pd.Timestamp('20130102'),
                    'C' : pd.Series(1,index=list(range(4)),dtype='float32'),
                    'D' : np.array([3] * 4, dtype='int32'),
                    'E' : pd.Categorical(["test", "train","test", "train"]),
                    'F' : 'foo'})
df2

A B C D E F
0 1.0 2013-01-02 1.0 3 test foo
1 1.0 2013-01-02 1.0 3 train foo
2 1.0 2013-01-02 1.0 3 test foo
3 1.0 2013-01-02 1.0 3 train foo 

  <script>
    const buttonEl =
      document.querySelector('#df-7bdbfccf-d22c-4fa9-afaf-b6f9252e8bbf button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-7bdbfccf-d22c-4fa9-afaf-b6f9252e8bbf');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 데이터프레임 결과물의 열은 다양한 데이터 타입 (dtypes)으로 구성됩니다.
1
df2.dtypes
A           float64
B    datetime64[ns]
C           float32
D             int32
E          category
F            object
dtype: object

  • IPython을 이용하고 계시다면 (공용 속성을 포함한) 열 이름에 대한 Tap 자동완성 기능이 자동으로 활성화 됩니다. 다음은 완성될 속성에 대한 부분집합 (subset)입니다.

  • 역자 주 : 아래 제시된 코드의 경우, IPython이 아닌 환경 (Google Colaboratory, Jupyter 등)에서는 사용이 불가능한 코드인 점에 주의하세요.

2. Viewing Data(데이터 확인하기)

1
2
print(df.tail(3)) # 끝에서 마지막 3줄을 불러옴
print(df.tail())  # 끝에서 마지막 5줄 불러옴
                   A         B         C         D
2013-01-04 -0.591139  1.840492  1.067977  1.738770
2013-01-05 -0.157689 -0.352707 -0.331992  0.634488
2013-01-06 -1.640009 -0.620559 -1.613156 -2.163666
                   A         B         C         D
2013-01-02 -1.099834 -1.311782  1.250473 -0.149189
2013-01-03 -0.348645 -0.913132  0.087372 -0.643829
2013-01-04 -0.591139  1.840492  1.067977  1.738770
2013-01-05 -0.157689 -0.352707 -0.331992  0.634488
2013-01-06 -1.640009 -0.620559 -1.613156 -2.163666

1
df.head() # 위에서 5줄 불러옴

A B C D
2013-01-01 0.300728 -0.263258 0.231729 -0.586384
2013-01-02 -1.099834 -1.311782 1.250473 -0.149189
2013-01-03 -0.348645 -0.913132 0.087372 -0.643829
2013-01-04 -0.591139 1.840492 1.067977 1.738770
2013-01-05 -0.157689 -0.352707 -0.331992 0.634488 

  <script>
    const buttonEl =
      document.querySelector('#df-c8c15f14-8937-456e-80ef-1223b8608eef button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-c8c15f14-8937-456e-80ef-1223b8608eef');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 인덱스 (index), 열 (column) 그리고 numpy 데이터에 대한 세부 정보를 봅니다.
1
df.index
DatetimeIndex(['2013-01-01', '2013-01-02', '2013-01-03', '2013-01-04',
               '2013-01-05', '2013-01-06'],
              dtype='datetime64[ns]', freq='D')

1
df.columns
Index(['A', 'B', 'C', 'D'], dtype='object')

1
df.values
array([[ 0.30072817, -0.26325765,  0.23172949, -0.58638441],
       [-1.09983384, -1.31178153,  1.25047287, -0.14918936],
       [-0.3486452 , -0.91313229,  0.08737214, -0.6438286 ],
       [-0.59113876,  1.84049219,  1.06797729,  1.73876959],
       [-0.15768942, -0.35270749, -0.33199219,  0.6344876 ],
       [-1.64000873, -0.62055935, -1.61315579, -2.16366558]])
  • describe()는 데이터의 대략적인 통계적 정보 요약을 보여줍니다.
1
df.describe()

A B C D
count 6.000000 6.000000 6.000000 6.000000
mean -0.589431 -0.270158 0.115401 -0.194968
std 0.692963 1.102985 1.039055 1.316046
min -1.640009 -1.311782 -1.613156 -2.163666
25% -0.972660 -0.839989 -0.227151 -0.629468
50% -0.469892 -0.486633 0.159551 -0.367787
75% -0.205428 -0.285620 0.858915 0.438568
max 0.300728 1.840492 1.250473 1.738770 

  <script>
    const buttonEl =
      document.querySelector('#df-e0954699-6177-4380-af7a-a41cf118eb02 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-e0954699-6177-4380-af7a-a41cf118eb02');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 데이터를 전치합니다.
1
df.T

2013-01-01 2013-01-02 2013-01-03 2013-01-04 2013-01-05 2013-01-06
A 0.300728 -1.099834 -0.348645 -0.591139 -0.157689 -1.640009
B -0.263258 -1.311782 -0.913132 1.840492 -0.352707 -0.620559
C 0.231729 1.250473 0.087372 1.067977 -0.331992 -1.613156
D -0.586384 -0.149189 -0.643829 1.738770 0.634488 -2.163666 

  <script>
    const buttonEl =
      document.querySelector('#df-0bfe1c42-a2e3-45ce-85a6-f9d827483b84 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-0bfe1c42-a2e3-45ce-85a6-f9d827483b84');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 축 별로 정렬합니다.
1
df.sort_index(axis=1, ascending=False)

D C B A
2013-01-01 -0.586384 0.231729 -0.263258 0.300728
2013-01-02 -0.149189 1.250473 -1.311782 -1.099834
2013-01-03 -0.643829 0.087372 -0.913132 -0.348645
2013-01-04 1.738770 1.067977 1.840492 -0.591139
2013-01-05 0.634488 -0.331992 -0.352707 -0.157689
2013-01-06 -2.163666 -1.613156 -0.620559 -1.640009 

  <script>
    const buttonEl =
      document.querySelector('#df-41da6cb4-f979-4360-b434-78445763e479 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-41da6cb4-f979-4360-b434-78445763e479');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

값 별로 정렬합니다.

1
df.sort_values(by='B')

A B C D
2013-01-02 -1.099834 -1.311782 1.250473 -0.149189
2013-01-03 -0.348645 -0.913132 0.087372 -0.643829
2013-01-06 -1.640009 -0.620559 -1.613156 -2.163666
2013-01-05 -0.157689 -0.352707 -0.331992 0.634488
2013-01-01 0.300728 -0.263258 0.231729 -0.586384
2013-01-04 -0.591139 1.840492 1.067977 1.738770 

  <script>
    const buttonEl =
      document.querySelector('#df-f495a21b-27c0-479d-8666-b36c68c7377e button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-f495a21b-27c0-479d-8666-b36c68c7377e');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

3. Selection(선택)

Getting(데이터 얻기)

  • df.A 와 동일한 Series를 생성하는 단일 열을 선택합니다.
1
df['A']
2013-01-01    0.300728
2013-01-02   -1.099834
2013-01-03   -0.348645
2013-01-04   -0.591139
2013-01-05   -0.157689
2013-01-06   -1.640009
Freq: D, Name: A, dtype: float64
  • 행을 분할하는 [ ]를 통해 선택합니다.
1
df[0:3]

A B C D
2013-01-01 0.300728 -0.263258 0.231729 -0.586384
2013-01-02 -1.099834 -1.311782 1.250473 -0.149189
2013-01-03 -0.348645 -0.913132 0.087372 -0.643829 

  <script>
    const buttonEl =
      document.querySelector('#df-5eddacfa-3ec5-4481-aab9-46338ea6faae button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-5eddacfa-3ec5-4481-aab9-46338ea6faae');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

1
df['20130102':'20130104']

A B C D
2013-01-02 -1.099834 -1.311782 1.250473 -0.149189
2013-01-03 -0.348645 -0.913132 0.087372 -0.643829
2013-01-04 -0.591139 1.840492 1.067977 1.738770 

  <script>
    const buttonEl =
      document.querySelector('#df-66eab04d-22d8-4e5a-a15b-72a682ab923c button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-66eab04d-22d8-4e5a-a15b-72a682ab923c');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

Selection by Label (label을 통한 선택)

1
df.loc[dates[0]]
A    0.300728
B   -0.263258
C    0.231729
D   -0.586384
Name: 2013-01-01 00:00:00, dtype: float64
  • 라벨을 사용하여 여러 축 (의 데이터)을 얻습니다.
1
df.loc[:, ['A','B']]

A B
2013-01-01 0.300728 -0.263258
2013-01-02 -1.099834 -1.311782
2013-01-03 -0.348645 -0.913132
2013-01-04 -0.591139 1.840492
2013-01-05 -0.157689 -0.352707
2013-01-06 -1.640009 -0.620559 

  <script>
    const buttonEl =
      document.querySelector('#df-57d0309b-fbf8-4929-b398-0d60d67063ee button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-57d0309b-fbf8-4929-b398-0d60d67063ee');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 양쪽 종단점을 포함한 라벨 슬라이싱을 봅니다.
1
df.loc['20130102':'20130104', ['A','B']]

A B
2013-01-02 -1.099834 -1.311782
2013-01-03 -0.348645 -0.913132
2013-01-04 -0.591139 1.840492 

  <script>
    const buttonEl =
      document.querySelector('#df-efba182b-23cd-4e19-bcb3-737481b2322b button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-efba182b-23cd-4e19-bcb3-737481b2322b');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 반환되는 객체의 차원를 줄입니다.
1
df.loc['20130102',['A','B']]
A   -1.099834
B   -1.311782
Name: 2013-01-02 00:00:00, dtype: float64
  • 스칼라 값을 얻습니다.
1
df.loc[dates[0],'A']
0.30072817102461075
  • 스칼라 값을 더 빠르게 구하는 방법입니다 (앞선 메소드와 동일합니다).
1
df.at[dates[0], 'A']
0.30072817102461075

Selection by Position(위치로 선택하기)

  • 넘겨받은 정수의 위치를 기준으로 선택합니다.
1
df.iloc[3]
A   -0.591139
B    1.840492
C    1.067977
D    1.738770
Name: 2013-01-04 00:00:00, dtype: float64
  • 정수로 표기된 슬라이스들을 통해, numpy / python과 유사하게 작동합니다.
1
df.iloc[3:5,0:2]

A B
2013-01-04 -0.591139 1.840492
2013-01-05 -0.157689 -0.352707 

  <script>
    const buttonEl =
      document.querySelector('#df-c542aa0e-249b-4629-a46b-89bdb9af6fed button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-c542aa0e-249b-4629-a46b-89bdb9af6fed');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 정수로 표기된 위치값의 리스트들을 통해, numpy / python의 스타일과 유사해집니다.
1
df.iloc[[1,2,4],[0,2]]

A C
2013-01-02 -1.099834 1.250473
2013-01-03 -0.348645 0.087372
2013-01-05 -0.157689 -0.331992 

  <script>
    const buttonEl =
      document.querySelector('#df-55d5daf9-c009-4d79-b075-b29d55365755 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-55d5daf9-c009-4d79-b075-b29d55365755');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 명시적으로 행을 나누고자 하는 경우입니다.
1
df.iloc[1:3,:]

A B C D
2013-01-02 -1.099834 -1.311782 1.250473 -0.149189
2013-01-03 -0.348645 -0.913132 0.087372 -0.643829 

  <script>
    const buttonEl =
      document.querySelector('#df-71b47cbb-5679-42a1-a3c2-9a4c0d93c913 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-71b47cbb-5679-42a1-a3c2-9a4c0d93c913');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 명시적으로 열을 나누고자 하는 경우입니다.
1
df.iloc[:,1:3]

B C
2013-01-01 -0.263258 0.231729
2013-01-02 -1.311782 1.250473
2013-01-03 -0.913132 0.087372
2013-01-04 1.840492 1.067977
2013-01-05 -0.352707 -0.331992
2013-01-06 -0.620559 -1.613156 

  <script>
    const buttonEl =
      document.querySelector('#df-9754adb0-b2e1-418c-b56a-aeac9f8649ee button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-9754adb0-b2e1-418c-b56a-aeac9f8649ee');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 명시적으로 (특정한) 값을 얻고자 하는 경우입니다.
1
df.iloc[1,1]
-1.311781527749884
  • 스칼라 값을 빠르게 얻는 방법입니다 (위의 방식과 동일합니다).
1
df.iat[1,1]
-1.311781527749884

Boolean Indexing

  • 데이터를 선택하기 위해 단일 열의 값을 사용합니다.
1
df[df.A > 0]

A B C D
2013-01-01 0.300728 -0.263258 0.231729 -0.586384 

  <script>
    const buttonEl =
      document.querySelector('#df-16044eaf-c404-4801-9af5-c6041cce22c2 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-16044eaf-c404-4801-9af5-c6041cce22c2');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • Boolean 조건을 충족하는 데이터프레임에서 값을 선택합니다.
1
df[df > 0]

A B C D
2013-01-01 0.300728 NaN 0.231729 NaN
2013-01-02 NaN NaN 1.250473 NaN
2013-01-03 NaN NaN 0.087372 NaN
2013-01-04 NaN 1.840492 1.067977 1.738770
2013-01-05 NaN NaN NaN 0.634488
2013-01-06 NaN NaN NaN NaN 

  <script>
    const buttonEl =
      document.querySelector('#df-31e35218-2394-43e7-bc03-233c098097df button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-31e35218-2394-43e7-bc03-233c098097df');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 필터링을 위한 메소드 isin()을 사용합니다.
1
df2 = df.copy()
1
2
df2['E'] = ['one', 'one', 'two', 'three', 'four', 'three']
df2

A B C D E
2013-01-01 0.300728 -0.263258 0.231729 -0.586384 one
2013-01-02 -1.099834 -1.311782 1.250473 -0.149189 one
2013-01-03 -0.348645 -0.913132 0.087372 -0.643829 two
2013-01-04 -0.591139 1.840492 1.067977 1.738770 three
2013-01-05 -0.157689 -0.352707 -0.331992 0.634488 four
2013-01-06 -1.640009 -0.620559 -1.613156 -2.163666 three 

  <script>
    const buttonEl =
      document.querySelector('#df-332f267a-157c-4418-b60d-3d784156a52c button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-332f267a-157c-4418-b60d-3d784156a52c');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

1
df2[df2['E'].isin(['two','four'])]

A B C D E
2013-01-03 -0.348645 -0.913132 0.087372 -0.643829 two
2013-01-05 -0.157689 -0.352707 -0.331992 0.634488 four 

  <script>
    const buttonEl =
      document.querySelector('#df-b0d3f1bc-76c3-42fa-80ac-b8f6838eae60 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-b0d3f1bc-76c3-42fa-80ac-b8f6838eae60');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

setting(설정)

  • 새 열을 설정하면 데이터가 인덱스 별로 자동 정렬됩니다.
1
2
s1 = pd.Series([1,2,3,4,5,6], index=pd.date_range('20130102', periods=6))
s1
2013-01-02    1
2013-01-03    2
2013-01-04    3
2013-01-05    4
2013-01-06    5
2013-01-07    6
Freq: D, dtype: int64

1
df['F'] = s1
  • 라벨에 의해 값을 설정합니다.
1
df.at[dates[0],'A'] = 0
  • 위치에 의해 값을 설정합니다.
1
df.iat[0,1] = 0
  • Numpy 배열을 사용한 할당에 의해 값을 설정합니다.
1
2
df.loc[:, 'D'] = np.array([5] * len(df))
df

A B C D F
2013-01-01 0.000000 0.000000 0.231729 5 NaN
2013-01-02 -1.099834 -1.311782 1.250473 5 1.0
2013-01-03 -0.348645 -0.913132 0.087372 5 2.0
2013-01-04 -0.591139 1.840492 1.067977 5 3.0
2013-01-05 -0.157689 -0.352707 -0.331992 5 4.0
2013-01-06 -1.640009 -0.620559 -1.613156 5 5.0 

  <script>
    const buttonEl =
      document.querySelector('#df-53de9a37-f969-4912-ac04-86a4cc6f332b button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-53de9a37-f969-4912-ac04-86a4cc6f332b');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • where 연산을 설정합니다.
1
df2 = df.copy()
1
2
df2[df2 > 0] = -df2
df2

A B C D F
2013-01-01 0.000000 0.000000 -0.231729 -5 NaN
2013-01-02 -1.099834 -1.311782 -1.250473 -5 -1.0
2013-01-03 -0.348645 -0.913132 -0.087372 -5 -2.0
2013-01-04 -0.591139 -1.840492 -1.067977 -5 -3.0
2013-01-05 -0.157689 -0.352707 -0.331992 -5 -4.0
2013-01-06 -1.640009 -0.620559 -1.613156 -5 -5.0 

  <script>
    const buttonEl =
      document.querySelector('#df-6b2a97e6-7a40-4d52-b4b0-11fa99013880 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-6b2a97e6-7a40-4d52-b4b0-11fa99013880');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

Missing Data(결측치)

  • Pandas는 결측치를 표현하기 위해 주로 np.nan 값을 사용합니다.
  • Reindexing으로 지정된 축 상의 인덱스를 변경 / 추가 / 삭제할 수 있습니다. Reindexing은 데이터의 복사본을 반환합니다.
1
df1 = df.reindex(index=dates[0:4], columns=list(df.columns) + ['E'])
1
2
df1.loc[dates[0]:dates[1], 'E'] =1
df1

A B C D F E
2013-01-01 0.000000 0.000000 0.231729 5 NaN 1.0
2013-01-02 -1.099834 -1.311782 1.250473 5 1.0 1.0
2013-01-03 -0.348645 -0.913132 0.087372 5 2.0 NaN
2013-01-04 -0.591139 1.840492 1.067977 5 3.0 NaN 

  <script>
    const buttonEl =
      document.querySelector('#df-854eff28-d5c6-493e-9c56-3d573cd836da button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-854eff28-d5c6-493e-9c56-3d573cd836da');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 결측치를 가지고 있는 행들을 지웁니다.
1
df1.dropna(how='any')

A B C D F E
2013-01-02 -1.099834 -1.311782 1.250473 5 1.0 1.0 

  <script>
    const buttonEl =
      document.querySelector('#df-9a8f93a0-529a-45d2-9bc3-6a5becc0ae58 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-9a8f93a0-529a-45d2-9bc3-6a5becc0ae58');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 결측치를 채워 넣습니다.
1
df1.fillna(value=5) # 5로 채워넣기

A B C D F E
2013-01-01 0.000000 0.000000 0.231729 5 5.0 1.0
2013-01-02 -1.099834 -1.311782 1.250473 5 1.0 1.0
2013-01-03 -0.348645 -0.913132 0.087372 5 2.0 5.0
2013-01-04 -0.591139 1.840492 1.067977 5 3.0 5.0 

  <script>
    const buttonEl =
      document.querySelector('#df-15ea5efe-2279-441d-917a-153c8e4accb6 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-15ea5efe-2279-441d-917a-153c8e4accb6');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • nan인 값에 boolean을 통한 표식을 얻습니다.
  • 역자 주 : 데이터프레임의 모든 값이 boolean 형태로 표시되도록 하며, nan인 값에만 True가 표시되게 하는 함수입니다.
1
pd.isna(df1)

A B C D F E
2013-01-01 False False False False True False
2013-01-02 False False False False False False
2013-01-03 False False False False False True
2013-01-04 False False False False False True 

  <script>
    const buttonEl =
      document.querySelector('#df-17e39b79-8d76-47c6-8375-a8f56909c185 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-17e39b79-8d76-47c6-8375-a8f56909c185');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

Operation (연산)

stats (통계)

  • 일반적으로 결측치를 제외한 후 연산됩니다.

  • 기술통계를 수행합니다.

1
df.mean()
A   -0.639553
B   -0.226281
C    0.115401
D    5.000000
F    3.000000
dtype: float64
  • 다른 축에서 동일한 연산을 수행합니다.
1
df.mean(1)
2013-01-01    1.307932
2013-01-02    0.967772
2013-01-03    1.165119
2013-01-04    2.063466
2013-01-05    1.631522
2013-01-06    1.225255
Freq: D, dtype: float64

  • 정렬이 필요하며, 차원이 다른 객체로 연산해보겠습니다. 또한, pandas는 지정된 차원을 따라 자동으로 브로드 캐스팅됩니다.

  • 역자 주 : broadcast란 numpy에서 유래한 용어로, n차원이나 스칼라 값으로 연산을 수행할 때 도출되는 결과의 규칙을 설명하는 것을 의미합니다.

1
2
s = pd.Series([1,3,5,np.nan,6,8], index=dates).shift(2)
s
2013-01-01    NaN
2013-01-02    NaN
2013-01-03    1.0
2013-01-04    3.0
2013-01-05    5.0
2013-01-06    NaN
Freq: D, dtype: float64

1
df.sub(s, axis='index')

A B C D F
2013-01-01 NaN NaN NaN NaN NaN
2013-01-02 NaN NaN NaN NaN NaN
2013-01-03 -1.348645 -1.913132 -0.912628 4.0 1.0
2013-01-04 -3.591139 -1.159508 -1.932023 2.0 0.0
2013-01-05 -5.157689 -5.352707 -5.331992 0.0 -1.0
2013-01-06 NaN NaN NaN NaN NaN 

  <script>
    const buttonEl =
      document.querySelector('#df-961448ec-241d-4b2b-bf3e-2b8c107e60b3 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-961448ec-241d-4b2b-bf3e-2b8c107e60b3');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

Apply(적용)

  • 데이터에 함수를 적용합니다.
1
df.apply(np.cumsum)

A B C D F
2013-01-01 0.000000 0.000000 0.231729 5 NaN
2013-01-02 -1.099834 -1.311782 1.482202 10 1.0
2013-01-03 -1.448479 -2.224914 1.569575 15 3.0
2013-01-04 -2.039618 -0.384422 2.637552 20 6.0
2013-01-05 -2.197307 -0.737129 2.305560 25 10.0
2013-01-06 -3.837316 -1.357688 0.692404 30 15.0 

  <script>
    const buttonEl =
      document.querySelector('#df-5fa85d8e-b443-4fcf-a35d-d5a22c845e81 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-5fa85d8e-b443-4fcf-a35d-d5a22c845e81');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

1
df.apply(lambda x: x.max() - x.min())
A    1.640009
B    3.152274
C    2.863629
D    0.000000
F    4.000000
dtype: float64

Hisrogrammin (히스토그래밍)

1
2
s = pd.Series(np.random.randint(0, 7, size=10))
s
0    2
1    6
2    2
3    2
4    4
5    3
6    4
7    4
8    3
9    5
dtype: int64

1
s.value_counts()
2    3
4    3
3    2
6    1
5    1
dtype: int64

string Methods(문자열 메소드)

  • Series는 다음의 코드와 같이 문자열 처리 메소드 모음 (set)을 가지고 있습니다.
    이 모음은 배열의 각 요소를 쉽게 조작할 수 있도록 만들어주는 문자열의 속성에 포함되어 있습니다.

  • 문자열의 패턴 일치 확인은 기본적으로 정규 표현식을 사용하며, 몇몇 경우에는 항상 정규 표현식을 사용함에 유의하십시오.

1
2
s = pd.Series(['A', 'B', 'C', 'AaBa', 'Baca', np.nan, 'CABA', 'dog', 'cat'])
s.str.lower()
0       a
1       b
2       c
3    aaba
4    baca
5     NaN
6    caba
7     dog
8     cat
dtype: object

Merge (병합)

Concat (연결)

  • 결합 (join) / 병합 (merge) 형태의 연산에 대한 인덱스, 관계 대수 기능을 위한 다양한 형태의 논리를 포함한 Series, 데이터프레임, Panel 객체를 손쉽게 결합할 수 있도록 하는 다양한 기능을 pandas 에서 제공합니다.
1
2
df = pd.DataFrame(np.random.randn(10, 4))
df

0 1 2 3
0 1.178802 1.240268 0.060703 -2.452726
1 -0.916616 -0.699856 -2.644101 -0.649991
2 -0.379350 0.733153 1.738607 2.509139
3 0.767562 0.810325 1.201008 0.163146
4 0.605380 -1.187634 0.672423 0.936118
5 -0.440754 -0.039716 0.420964 0.054439
6 0.651187 -1.113766 0.354955 -0.271147
7 1.874887 -1.369062 -0.033655 -0.506732
8 0.921916 -0.950195 -0.304002 2.024843
9 0.038615 2.242273 -1.858805 -0.206487 

  <script>
    const buttonEl =
      document.querySelector('#df-6013bb9c-e791-4cbd-ae7f-70916e5330bc button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-6013bb9c-e791-4cbd-ae7f-70916e5330bc');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

1
2
3
# break it into pieces
pieces = [df[:3], df[3:7], df[7:]]
pd.concat(pieces)

0 1 2 3
0 1.178802 1.240268 0.060703 -2.452726
1 -0.916616 -0.699856 -2.644101 -0.649991
2 -0.379350 0.733153 1.738607 2.509139
3 0.767562 0.810325 1.201008 0.163146
4 0.605380 -1.187634 0.672423 0.936118
5 -0.440754 -0.039716 0.420964 0.054439
6 0.651187 -1.113766 0.354955 -0.271147
7 1.874887 -1.369062 -0.033655 -0.506732
8 0.921916 -0.950195 -0.304002 2.024843
9 0.038615 2.242273 -1.858805 -0.206487 

  <script>
    const buttonEl =
      document.querySelector('#df-0f103399-f83b-4a1a-a989-947d1f863afe button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-0f103399-f83b-4a1a-a989-947d1f863afe');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

join(결합)

  • SQL 방식으로 병합합니다.
1
2
3
left = pd.DataFrame({'key': ['foo', 'foo'], 'lval': [1, 2]})
right = pd.DataFrame({'key': ['foo', 'foo'], 'rval': [4, 5]})
left

key lval
0 foo 1
1 foo 2 

  <script>
    const buttonEl =
      document.querySelector('#df-33c0a123-0496-477d-9dbf-77cb03dd8f02 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-33c0a123-0496-477d-9dbf-77cb03dd8f02');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

1
right

key rval
0 foo 4
1 foo 5 

  <script>
    const buttonEl =
      document.querySelector('#df-8d898d93-29c2-4b09-8537-80ba4b84dba6 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-8d898d93-29c2-4b09-8537-80ba4b84dba6');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

1
pd.merge(left, right, on= 'key')

key lval rval
0 foo 1 4
1 foo 1 5
2 foo 2 4
3 foo 2 5 

  <script>
    const buttonEl =
      document.querySelector('#df-7417a0f2-233d-4ff9-9215-691582433e92 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-7417a0f2-233d-4ff9-9215-691582433e92');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 다른 예시입니다.
1
2
3
4
left = pd.DataFrame({'key' : ['foo', 'bar'], 'lval' : [1, 2]})
right = pd.DataFrame({'key': ['foo', 'bar'], 'rval': [4, 5]})
left

key lval
0 foo 1
1 bar 2 

  <script>
    const buttonEl =
      document.querySelector('#df-83539243-821e-4da6-bba3-31cb6ede60da button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-83539243-821e-4da6-bba3-31cb6ede60da');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

1
right

key rval
0 foo 4
1 bar 5 

  <script>
    const buttonEl =
      document.querySelector('#df-a227dc5d-7989-447c-9111-3ed378e30d1e button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-a227dc5d-7989-447c-9111-3ed378e30d1e');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

1
pd.merge(left, right, on= 'key')

key lval rval
0 foo 1 4
1 bar 2 5 

  <script>
    const buttonEl =
      document.querySelector('#df-10bb1151-5184-40e9-a764-cc2a8c0e1d47 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-10bb1151-5184-40e9-a764-cc2a8c0e1d47');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

Append (추가)

  • 데이터프레임에 행을 추가합니다.
1
2
df = pd.DataFrame(np.random.randn(8, 4), columns=['A', 'B', 'C', 'D'])
df

A B C D
0 -0.707810 -0.616206 0.557429 -0.673691
1 0.398359 0.590156 0.024199 1.396677
2 -0.183290 0.047769 0.779775 1.442445
3 0.084316 -1.308026 -0.809909 -0.100735
4 -0.511133 -0.380242 -1.043381 -0.806634
5 -0.580510 -0.395366 0.717878 -0.685339
6 -1.166817 0.761797 -0.346222 1.487303
7 -1.198588 -0.761424 1.893708 1.162279 

  <script>
    const buttonEl =
      document.querySelector('#df-247e1446-ced9-4766-87ac-8f6a6c04247e button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-247e1446-ced9-4766-87ac-8f6a6c04247e');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

1
2
s = df.iloc[3]
s
A    0.084316
B   -1.308026
C   -0.809909
D   -0.100735
Name: 3, dtype: float64

1
df.append(s, ignore_index=True)

A B C D
0 -0.707810 -0.616206 0.557429 -0.673691
1 0.398359 0.590156 0.024199 1.396677
2 -0.183290 0.047769 0.779775 1.442445
3 0.084316 -1.308026 -0.809909 -0.100735
4 -0.511133 -0.380242 -1.043381 -0.806634
5 -0.580510 -0.395366 0.717878 -0.685339
6 -1.166817 0.761797 -0.346222 1.487303
7 -1.198588 -0.761424 1.893708 1.162279
8 0.084316 -1.308026 -0.809909 -0.100735 

  <script>
    const buttonEl =
      document.querySelector('#df-18f32566-d3d3-4b9d-bc46-7260ec1ca6ca button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-18f32566-d3d3-4b9d-bc46-7260ec1ca6ca');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

Grouping (그룹화)

-그룹화는 다음 단계 중 하나 이상을 포함하는 과정을 가리킵니다.

  • 몇몇 기준에 따라 여러 그룹으로 데이터를 분할 (splitting)
  • 각 그룹에 독립적으로 함수를 적용 (applying)
  • 결과물들을 하나의 데이터 구조로 결합 (combining)
1
2
3
4
5
6
7
8
df = pd.DataFrame(
    {
        'A' : ['foo', 'bar', 'foo', 'bar', 'foo', 'bar', 'foo', 'foo'],
        'B' : ['one', 'one', 'two', 'three', 'two', 'two', 'one', 'three'],
        'C' : np.random.randn(8),
        'D' : np.random.randn(8)
    })
df

A B C D
0 foo one 1.069249 0.365181
1 bar one 0.137894 -0.394584
2 foo two -1.473601 0.771336
3 bar three -0.026117 0.153736
4 foo two 0.675027 0.977329
5 bar two -0.396978 -0.150105
6 foo one 1.017942 1.533993
7 foo three -1.410921 -0.479321 

  <script>
    const buttonEl =
      document.querySelector('#df-b2b359ca-0618-47e5-b037-187352f6d190 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-b2b359ca-0618-47e5-b037-187352f6d190');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 생성된 데이터프레임을 그룹화한 후 각 그룹에 sum() 함수를 적용합니다.
1
df.groupby('A').sum()

C D
A
bar -0.285202 -0.390953
foo -0.122304 3.168518 

  <script>
    const buttonEl =
      document.querySelector('#df-d1705048-d15c-45e1-bfd0-a87c1641a436 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-d1705048-d15c-45e1-bfd0-a87c1641a436');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 여러 열을 기준으로 그룹화하면 계층적 인덱스가 형성됩니다. 여기에도 sum 함수를 적용할 수 있습니다.
1
df.groupby(['A', 'B']).sum()

C D
A B
bar one 0.137894 -0.394584
three -0.026117 0.153736
two -0.396978 -0.150105
foo one 2.087191 1.899175
three -1.410921 -0.479321
two -0.798575 1.748664 

  <script>
    const buttonEl =
      document.querySelector('#df-8697c255-46f3-4c53-bd92-5ce4e549ba97 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-8697c255-46f3-4c53-bd92-5ce4e549ba97');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

Reshaping(변형)

Stack(스택)

1
2
3
4
tuples = list(zip(*[['bar', 'bar', 'baz', 'baz',
                     'foo', 'foo', 'qux', 'qux'],
                    ['one', 'two', 'one', 'two',
                     'one', 'two', 'one', 'two']]))
1
2
3
4
index = pd.MultiIndex.from_tuples(tuples, names=['first', 'second'])
df = pd.DataFrame(np.random.randn(8, 2), index=index, columns=['A', 'B'])
df2 = df[:4]
df

A B
first second
bar one 0.046976 0.725962
two -0.368482 -0.562111
baz one 1.175016 -0.150060
two -0.494980 0.665989
foo one 1.328767 -0.932962
two 0.192983 1.109156
qux one -0.421099 -0.253088
two -0.872046 -1.090497 

  <script>
    const buttonEl =
      document.querySelector('#df-4c495b34-0698-4c12-b61e-df5d63669b09 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-4c495b34-0698-4c12-b61e-df5d63669b09');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • stack() 메소드는 데이터프레임 열들의 계층을 “압축”합니다.
1
2
stacked = df2.stack()
stacked
first  second   
bar    one     A    0.046976
               B    0.725962
       two     A   -0.368482
               B   -0.562111
baz    one     A    1.175016
               B   -0.150060
       two     A   -0.494980
               B    0.665989
dtype: float64
  • “Stack된” 데이터프레임 또는 (MultiIndex를 인덱스로 사용하는) Series인 경우, stack()의 역 연산은 unstack()이며, 기본적으로 마지막 계층을 unstack합니다.
1
stacked.unstack()

A B
first second
bar one 0.046976 0.725962
two -0.368482 -0.562111
baz one 1.175016 -0.150060
two -0.494980 0.665989 

  <script>
    const buttonEl =
      document.querySelector('#df-2a8b2ec1-992e-4699-8e27-b6263f7149bd button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-2a8b2ec1-992e-4699-8e27-b6263f7149bd');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

1
stacked.unstack(1)

second one two
first
bar A 0.046976 -0.368482
B 0.725962 -0.562111
baz A 1.175016 -0.494980
B -0.150060 0.665989 

  <script>
    const buttonEl =
      document.querySelector('#df-5f2284f0-d6dc-4fc6-8af0-f7212cf11085 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-5f2284f0-d6dc-4fc6-8af0-f7212cf11085');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

1
stacked.unstack(0)

first bar baz
second
one A 0.046976 1.175016
B 0.725962 -0.150060
two A -0.368482 -0.494980
B -0.562111 0.665989 

  <script>
    const buttonEl =
      document.querySelector('#df-bbf1c47c-401a-4319-935f-677ccff31631 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-bbf1c47c-401a-4319-935f-677ccff31631');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

Pivot Tables(피벗 테이블)

1
2
3
4
5
6
df = pd.DataFrame({'A' : ['one', 'one', 'two', 'three'] * 3,
                   'B' : ['A', 'B', 'C'] * 4,
                   'C' : ['foo', 'foo', 'foo', 'bar', 'bar', 'bar'] * 2,
                   'D' : np.random.randn(12),
                   'E' : np.random.randn(12)})
df

A B C D E
0 one A foo 0.561255 0.791926
1 one B foo -0.401628 -0.185673
2 two C foo 0.184004 1.424850
3 three A bar 1.448154 1.572973
4 one B bar 0.213895 -1.251325
5 one C bar 0.135554 -0.691501
6 two A foo -0.329284 -1.046691
7 three B foo 0.921972 0.967578
8 one C foo 0.215366 -0.041228
9 one A bar 0.161393 -1.637091
10 two B bar 0.561090 1.233453
11 three C bar -0.513841 -1.183525 

  <script>
    const buttonEl =
      document.querySelector('#df-fe9d9c6b-c4d1-4e2d-9e73-d3c4dc01a1b4 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-fe9d9c6b-c4d1-4e2d-9e73-d3c4dc01a1b4');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 이 데이터로부터 피봇 테이블을 매우 쉽게 생성할 수 있습니다.
1
pd.pivot_table(df, values='D', index=['A', 'B'], columns=['C'])

C bar foo
A B
one A 0.161393 0.561255
B 0.213895 -0.401628
C 0.135554 0.215366
three A 1.448154 NaN
B NaN 0.921972
C -0.513841 NaN
two A NaN -0.329284
B 0.561090 NaN
C NaN 0.184004 

  <script>
    const buttonEl =
      document.querySelector('#df-c3f2f893-1f74-45ed-8a1a-858cb444e62b button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-c3f2f893-1f74-45ed-8a1a-858cb444e62b');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

Times Seires (시계열)

  • Pandas는 자주 일어나는 변환 (예시 : 5분마다 일어나는 데이터에 대한 2차 데이터 변환) 사이에 수행하는 리샘플링 연산을 위한 간단하고, 강력하며, 효율적인 함수를 제공합니다. 이는 재무 (금융) 응용에서 매우 일반적이지만 이에 국한되지는 않습니다.
1
2
3
rng = pd.date_range('1/1/2012', periods=100, freq='S')
ts = pd.Series(np.random.randint(0, 500, len(rng)), index=rng)
ts.resample('5Min').sum()
2012-01-01    26641
Freq: 5T, dtype: int64
  • 시간대를 표현합니다.
1
2
3
rng = pd.date_range('3/6/2012 00:00', periods=5, freq='D')
ts = pd.Series(np.random.randn(len(rng)), rng)
ts
2012-03-06    0.581358
2012-03-07   -0.835184
2012-03-08   -1.291719
2012-03-09    0.349362
2012-03-10   -1.415495
Freq: D, dtype: float64

1
2
ts_utc = ts.tz_localize('UTC')
ts_utc
2012-03-06 00:00:00+00:00    0.581358
2012-03-07 00:00:00+00:00   -0.835184
2012-03-08 00:00:00+00:00   -1.291719
2012-03-09 00:00:00+00:00    0.349362
2012-03-10 00:00:00+00:00   -1.415495
Freq: D, dtype: float64
  • 다른 시간대로 변환합니다.
1
ts_utc.tz_convert('US/Eastern')
2012-03-05 19:00:00-05:00    0.581358
2012-03-06 19:00:00-05:00   -0.835184
2012-03-07 19:00:00-05:00   -1.291719
2012-03-08 19:00:00-05:00    0.349362
2012-03-09 19:00:00-05:00   -1.415495
Freq: D, dtype: float64
  • 시간 표현 ↔ 기간 표현으로 변환합니다.
1
2
3
rng = pd.date_range('1/1/2012', periods=5, freq='M')
ts = pd.Series(np.random.randn(len(rng)), index=rng)
ts
2012-01-31    0.748775
2012-02-29    0.516551
2012-03-31   -0.413149
2012-04-30    1.247230
2012-05-31   -1.076339
Freq: M, dtype: float64

1
2
ps = ts.to_period()
ps
2012-01    0.748775
2012-02    0.516551
2012-03   -0.413149
2012-04    1.247230
2012-05   -1.076339
Freq: M, dtype: float64

1
ps.to_timestamp()
2012-01-01    0.748775
2012-02-01    0.516551
2012-03-01   -0.413149
2012-04-01    1.247230
2012-05-01   -1.076339
Freq: MS, dtype: float64
  • 기간 ↔ 시간 변환은 편리한 산술 기능들을 사용할 수 있도록 만들어줍니다. 다음 예제에서, 우리는 11월에 끝나는 연말 결산의 분기별 빈도를 분기말 익월의 월말일 오전 9시로 변환합니다.
1
2
3
4
prng = pd.period_range('1990Q1', '2000Q4', freq='Q-NOV')
ts = pd.Series(np.random.randn(len(prng)), prng)
ts.index = (prng.asfreq('M', 'e') + 1).asfreq('H', 's') + 9
ts.head()
1990-03-01 09:00   -1.061704
1990-06-01 09:00   -0.079417
1990-09-01 09:00   -0.444862
1990-12-01 09:00   -1.855021
1991-03-01 09:00    1.837690
Freq: H, dtype: float64

Categoricals(범주화)

  • Pandas는 데이터프레임 내에 범주형 데이터를 포함할 수 있습니다.
1
2
df = pd.DataFrame({"id":[1,2,3,4,5,6], "raw_grade":['a','b','b','a','a','e']})
df

id raw_grade
0 1 a
1 2 b
2 3 b
3 4 a
4 5 a
5 6 e 

  <script>
    const buttonEl =
      document.querySelector('#df-e40d400c-1163-4773-a477-140c59e9c2e1 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-e40d400c-1163-4773-a477-140c59e9c2e1');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>
  • 가공하지 않은 성적을 범주형 데이터로 변환합니다.
1
2
df["grade"] = df["raw_grade"].astype("category")
df["grade"]
0    a
1    b
2    b
3    a
4    a
5    e
Name: grade, dtype: category
Categories (3, object): ['a', 'b', 'e']
  • 범주에 더 의미 있는 이름을 붙여주세요 (Series.cat.categories로 할당하는 것이 적합합니다).
1
2
df["grade"].cat.categories = ["very good", "good", "very bad"]

  • 범주의 순서를 바꾸고 동시에 누락된 범주를 추가합니다 (Series.cat에 속하는 메소드는 기본적으로 새로운 Series를 반환합니다).
1
2
df["grade"] = df["grade"].cat.set_categories(["very bad", "bad", "medium", "good", "very good"])
df["grade"]
0    very good
1         good
2         good
3    very good
4    very good
5     very bad
Name: grade, dtype: category
Categories (5, object): ['very bad', 'bad', 'medium', 'good', 'very good']

  • 정렬은 사전 순서가 아닌, 해당 범주에서 지정된 순서대로 배열합니다.

  • 역자 주 : 131번에서 very bad, bad, medium, good, very good 의 순서로 기재되어 있기 때문에 정렬 결과도 해당 순서대로 배열됩니다.

1
df.sort_values(by="grade")

id raw_grade grade
5 6 e very bad
1 2 b good
2 3 b good
0 1 a very good
3 4 a very good
4 5 a very good 

  <script>
    const buttonEl =
      document.querySelector('#df-1c2a0237-5699-4997-989a-33392ab7c550 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-1c2a0237-5699-4997-989a-33392ab7c550');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

1
df.groupby("grade").size()
grade
very bad     1
bad          0
medium       0
good         2
very good    3
dtype: int64

Plotting (그래프)

1
2
3
ts = pd.Series(np.random.randn(1000), index=pd.date_range('1/1/2000', periods=1000))
ts = ts.cumsum()
ts.plot()
<matplotlib.axes._subplots.AxesSubplot at 0x7f89ebda41d0>




/images/10minutes_to_pandas

png

  • 데이터프레임에서 plot() 메소드는 라벨이 존재하는 모든 열을 그릴 때 편리합니다.
1
2
3
4
df = pd.DataFrame(np.random.randn(1000, 4), index=ts.index,
                  columns=['A', 'B', 'C', 'D'])
df = df.cumsum()
plt.figure(); df.plot(); plt.legend(loc='best')
<matplotlib.legend.Legend at 0x7f89eb792990>




<Figure size 432x288 with 0 Axes>

png

Getting Data In / Out (데이터 입출력)

CSV

  • csv파일을 씁니다.
1
df.to_csv('foo.csv')

csv 파일을 읽습니다.

1
pd.read_csv('foo.csv')

Unnamed: 0 A B C D
0 2000-01-01 0.077785 1.354574 0.335250 -0.643291
1 2000-01-02 1.506306 0.603573 1.431830 -0.151375
2 2000-01-03 2.046989 -0.243843 1.469860 -1.276268
3 2000-01-04 4.195420 -0.137163 0.435910 -1.063562
4 2000-01-05 5.022651 -0.684153 -0.179983 0.833490
... ... ... ... ... ...
995 2002-09-22 60.234084 -33.177527 -12.221695 -38.068835
996 2002-09-23 60.599992 -32.577022 -13.140842 -38.394246
997 2002-09-24 60.739624 -30.809578 -13.287040 -38.570248
998 2002-09-25 60.622057 -31.091125 -13.027110 -39.217957
999 2002-09-26 62.868526 -31.140053 -12.690182 -39.383923

1000 rows × 5 columns

  <script>
    const buttonEl =
      document.querySelector('#df-09d8c0d4-bf7c-472f-813c-05a5e4ad76fe button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-09d8c0d4-bf7c-472f-813c-05a5e4ad76fe');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

HDF5

  • HDFStores에 읽고 씁니다.
  • HDF5 Store에 씁니다.
1
df.to_hdf('foo.h5', 'df')
  • HDF5 Store에서 읽어옵니다.
1
pd.read_hdf('foo.h5', 'df')

A B C D
2000-01-01 0.077785 1.354574 0.335250 -0.643291
2000-01-02 1.506306 0.603573 1.431830 -0.151375
2000-01-03 2.046989 -0.243843 1.469860 -1.276268
2000-01-04 4.195420 -0.137163 0.435910 -1.063562
2000-01-05 5.022651 -0.684153 -0.179983 0.833490
... ... ... ... ...
2002-09-22 60.234084 -33.177527 -12.221695 -38.068835
2002-09-23 60.599992 -32.577022 -13.140842 -38.394246
2002-09-24 60.739624 -30.809578 -13.287040 -38.570248
2002-09-25 60.622057 -31.091125 -13.027110 -39.217957
2002-09-26 62.868526 -31.140053 -12.690182 -39.383923

1000 rows × 4 columns

  <script>
    const buttonEl =
      document.querySelector('#df-88b181fb-5386-46b5-93ff-12fa70dae0a4 button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-88b181fb-5386-46b5-93ff-12fa70dae0a4');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

Excel

  • MS Excel에 읽고 씁니다.
  • 엑셀 파일에 씁니다.
1
df.to_excel('foo.xlsx', sheet_name='Sheet1')
  • 엑셀 파일을 읽어옵니다
1
pd.read_excel('foo.xlsx', 'Sheet1', index_col=None, na_values=['NA'])

Unnamed: 0 A B C D
0 2000-01-01 0.077785 1.354574 0.335250 -0.643291
1 2000-01-02 1.506306 0.603573 1.431830 -0.151375
2 2000-01-03 2.046989 -0.243843 1.469860 -1.276268
3 2000-01-04 4.195420 -0.137163 0.435910 -1.063562
4 2000-01-05 5.022651 -0.684153 -0.179983 0.833490
... ... ... ... ... ...
995 2002-09-22 60.234084 -33.177527 -12.221695 -38.068835
996 2002-09-23 60.599992 -32.577022 -13.140842 -38.394246
997 2002-09-24 60.739624 -30.809578 -13.287040 -38.570248
998 2002-09-25 60.622057 -31.091125 -13.027110 -39.217957
999 2002-09-26 62.868526 -31.140053 -12.690182 -39.383923

1000 rows × 5 columns

  <script>
    const buttonEl =
      document.querySelector('#df-ceafdb00-9193-41c5-910f-15ff5b0d4b1b button.colab-df-convert');
    buttonEl.style.display =
      google.colab.kernel.accessAllowed ? 'block' : 'none';

    async function convertToInteractive(key) {
      const element = document.querySelector('#df-ceafdb00-9193-41c5-910f-15ff5b0d4b1b');
      const dataTable =
        await google.colab.kernel.invokeFunction('convertToInteractive',
                                                 [key], {});
      if (!dataTable) return;

      const docLinkHtml = 'Like what you see? Visit the ' +
        '<a target="_blank" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'
        + ' to learn more about interactive tables.';
      element.innerHTML = '';
      dataTable['output_type'] = 'display_data';
      await google.colab.output.renderOutput(dataTable, element);
      const docLink = document.createElement('div');
      docLink.innerHTML = docLinkHtml;
      element.appendChild(docLink);
    }
  </script>
</div>

Gotchas (잡았다!)

  • 연산 수행시 다음과 같은 예외 상황을 볼 수 도 있습니다.
  • 이러한 경우에는 any(), all(), empty 등을 사용해서 무엇을 원하는지를 선택 (반영)해주어야 합니다.
1
2
if pd.Series([False, True, False])is not None:
  print("I was not None true")
I was not None true

10분 완성 Pandas

http://griotold.github.io/2022/03/25/10minutes_to_pandas/

Author

HS

Posted on

2022-03-25

Updated on

2022-03-25

Licensed under

You need to set install_url to use ShareThis. Please set it in _config.yml.

Like this article? Support the author with

You forgot to set the business or currency_code for Paypal. Please set it in _config.yml.

Comments

You forgot to set the shortname for Disqus. Please set it in _config.yml.
Follow

Links

Categories

Recents

Archives

Subscribe for updates

You need to set client_id and slot_id to show this AD unit. Please set it in _config.yml.

follow.it

×