概要

Pythonで業務システムからエクスポートしたShift-JISとされているファイルを処理する際、エンコーディングとしてcp932を使用することが推奨されます。これは、Shift-JISとcp932の間には微妙な違いがあり、特に日本語の一部の文字（一部の記号や特殊文字など）がShift-JISではなくcp932でしか正しく表現できないためです。そのため、エラーを避けるためには、Pythonでこれらのファイルを扱う際にはcp932として処理することが望ましいです。

㈱という特殊な文字をshift_jisエンコーディングで処理しようとすると、エラーが発生します。これは、shift_jisがこの特殊な文字をサポートしていないためです。このような問題を避けるためには、cp932エンコーディングの使用を検討してみてください。

import pandas as pd

data = [
    {"col_1": "㈱"},
]

df = pd.DataFrame(data)
df['col_1'] = df['col_1'].str.encode('shift_jis')
df

UnicodeEncodeError: 'shift_jis' codec can't encode character '\u3231' in position 0: illegal multibyte sequence

本記事にて、エラーとなるコードのサンプルと同様のエラーが発生する文字を記載します。

エラーとなるコードサンプル

ファイルとして書き込む場合

shift_jisではエラーとなってしまう。

# Importing Required Library
import os

# Creating test.csv with data to be written
data = """
col_1
㈱
""".strip()

# Creating and Writing Data to test.csv file in dbfs/test folder
with open("/dbfs/test.csv", "w", encoding='shift_jis') as file:
    file.write(data)

cp932に変更すると正常終了する。

# Importing Required Library
import os

# Creating test.csv with data to be written
data = """
col_1
㈱
""".strip()

# Creating and Writing Data to test.csv file in dbfs/test folder
with open("/dbfs/test.csv", "w", encoding='cp932') as file:
    file.write(data)

Pandas データフレームを作成する場合

shift_jisではエラーとなってしまう。

import pandas as pd

df = pd.read_csv("/dbfs/test.csv", encoding='shift_jis')
df

cp932に変更すると正常終了する。

import pandas as pd

df = pd.read_csv("/dbfs/test.csv", encoding='cp932')
df

Pandas データフレームにてカラムを encode する場合

shift_jisではエラーとなってしまう。

import pandas as pd

data = [
    {"col_1": "㈱"},
]

df = pd.DataFrame(data)
df['col_1'] = df['col_1'].str.encode('shift_jis')
df

cp932に変更すると正常終了する。

import pandas as pd

data = [
    {"col_1": "㈱"},
]

df = pd.DataFrame(data)
df['col_1'] = df['col_1'].str.encode('cp932')
df

同様のエラーが発生する文字列

下記の文字が含まれている場合には、shift_jisとして扱う場合にはエラーとなります。

col_1	col_1_encoded
①	h0A=
②	h0E=
③	h0I=
④	h0M=
⑤	h0Q=
⑥	h0U=
⑦	h0Y=
⑧	h0c=
⑨	h0g=
⑩	h0k=
⑪	h0o=
⑫	h0s=
⑬	h0w=
⑭	h00=
⑮	h04=
⑯	h08=
⑰	h1A=
⑱	h1E=
⑲	h1I=
⑳	h1M=
Ⅰ	h1Q=
Ⅱ	h1U=
Ⅲ	h1Y=
Ⅳ	h1c=
Ⅴ	h1g=
Ⅵ	h1k=
Ⅶ	h1o=
Ⅷ	h1s=
Ⅸ	h1w=
Ⅹ	h10=
ⅰ	7u8=
ⅱ	7vA=
ⅲ	7vE=
ⅳ	7vI=
ⅴ	7vM=
ⅵ	7vQ=
ⅶ	7vU=
ⅷ	7vY=
ⅸ	7vc=
ⅹ	7vg=
・	gUU=
㍉	h18=
㌔	h2A=
㌢	h2E=
㍍	h2I=
㌘	h2M=
㌧	h2Q=
㌃	h2U=
㌶	h2Y=
㍑	h2c=
㍗	h2g=
㌍	h2k=
㌦	h2o=
㌣	h2s=
㌫	h2w=
㍊	h20=
㌻	h24=
㎜	h28=
㎝	h3A=
㎞	h3E=
㎎	h3I=
㎏	h3M=
㏄	h3Q=
㎡	h3U=
㍻	h34=
〝	h4A=
〟	h4E=
№	h4I=
㏍	h4M=
℡	h4Q=
㊤	h4U=
㊥	h4Y=
㊦	h4c=
㊧	h4g=
㊨	h4k=
㈱	h4o=
㈲	h4s=
㈹	h4w=
㍾	h40=
㍽	h44=
㍼	h48=
≒	geA=
≡	gd8=
∫	gec=
∮	h5M=
∑	h5Q=
√	geM=
⊥	gds=
∠	gdo=
⊿	h5k=
∵	geY=
∩	gb8=
∪	gb4=
亜	iJ8=
唖	iKA=
娃	iKE=
阿	iKI=
哀	iKM=
愛	iKQ=
挨	iKU=
姶	iKY=
逢	iKc=
葵	iKg=
茜	iKk=
穐	iKo=
悪	iKs=
握	iKw=
渥	iK0=
旭	iK4=
葦	iK8=
￢	gco=
￤	7vo=
＇	7vs=
＂	7vw=
㈱	h4o=
№	h4I=
℡	h4Q=
∵	geY=
纊	7UA=
褜	7UE=
鍈	7UI=
銈	7UM=
蓜	7UQ=
俉	7UU=
炻	7UY=
昱	7Uc=
棈	7Ug=
鋹	7Uk=
曻	7Uo=
彅	7Us=
丨	7Uw=
仡	7U0=
仼	7U4=
伀	7U8=
伃	7VA=
伹	7VE=
佖	7VI=
侒	7VM=
侊	7VQ=
侚	7VU=
侔	7VY=
俍	7Vc=
偀	7Vg=
倢	7Vk=
俿	7Vo=
倞	7Vs=
偆	7Vw=
偰	7V0=
偂	7V4=
傔	7V8=
僴	7WA=
僘	7WE=
兊	7WI=
兤	7WM=
冝	7WQ=
冾	7WU=
凬	7WY=
刕	7Wc=
劜	7Wg=
劦	7Wk=
勀	7Wo=
勛	7Ws=
匀	7Ww=
匇	7W0=
匤	7W4=
卲	7W8=
厓	7XA=
厲	7XE=
叝	7XI=
﨎	7XM=
咜	7XQ=
咊	7XU=
咩	7XY=
哿	7Xc=
喆	7Xg=
坙	7Xk=
坥	7Xo=
垬	7Xs=
埈	7Xw=
埇	7X0=
﨏	7X4=
塚	7YA=
增	7YE=
墲	7YI=
夋	7YM=
奓	7YQ=
奛	7YU=
奝	7YY=
奣	7Yc=
妤	7Yg=
妺	7Yk=
孖	7Yo=
寀	7Ys=
甯	7Yw=
寘	7Y0=
寬	7Y4=
尞	7Y8=
岦	7ZA=
岺	7ZE=
峵	7ZI=
崧	7ZM=
嵓	7ZQ=
﨑	7ZU=
嵂	7ZY=
嵭	7Zc=
嶸	7Zg=
嶹	7Zk=
巐	7Zo=
弡	7Zs=
弴	7Zw=
彧	7Z0=
德	7Z4=
忞	7Z8=
恝	7aA=
悅	7aE=
悊	7aI=
惞	7aM=
惕	7aQ=
愠	7aU=
惲	7aY=
愑	7ac=
愷	7ag=
愰	7ak=
憘	7ao=
戓	7as=
抦	7aw=
揵	7a0=
摠	7a4=
撝	7a8=
擎	7bA=
敎	7bE=
昀	7bI=
昕	7bM=
昻	7bQ=
昉	7bU=
昮	7bY=
昞	7bc=
昤	7bg=
晥	7bk=
晗	7bo=
晙	7bs=
晴	7bw=
晳	7b0=
暙	7b4=
暠	7b8=
暲	7cA=
暿	7cE=
曺	7cI=
朎	7cM=
朗	7cQ=
杦	7cU=
枻	7cY=
桒	7cc=
柀	7cg=
栁	7ck=
桄	7co=
棏	7cs=
﨓	7cw=
楨	7c0=
﨔	7c4=
榘	7c8=
槢	7dA=
樰	7dE=
橫	7dI=
橆	7dM=
橳	7dQ=
橾	7dU=
櫢	7dY=
櫤	7dc=
毖	7dg=
氿	7dk=
汜	7do=
沆	7ds=
汯	7dw=
泚	7d0=
洄	7d4=
涇	7d8=
浯	7eA=
涖	7eE=
涬	7eI=
淏	7eM=
淸	7eQ=
淲	7eU=
淼	7eY=
渹	7ec=
湜	7eg=
渧	7ek=
渼	7eo=
溿	7es=
澈	7ew=
澵	7e0=
濵	7e4=
瀅	7e8=
瀇	7fA=
瀨	7fE=
炅	7fI=
炫	7fM=
焏	7fQ=
焄	7fU=
煜	7fY=
煆	7fc=
煇	7fg=
凞	7fk=
燁	7fo=
燾	7fs=
犱	7fw=
犾	7kA=
猤	7kE=
猪	7kI=
獷	7kM=
玽	7kQ=
珉	7kU=
珖	7kY=
珣	7kc=
珒	7kg=
琇	7kk=
珵	7ko=
琦	7ks=
琪	7kw=
琩	7k0=
琮	7k4=
瑢	7k8=
璉	7lA=
璟	7lE=
甁	7lI=
畯	7lM=
皂	7lQ=
皜	7lU=
皞	7lY=
皛	7lc=
皦	7lg=
益	7lk=
睆	7lo=
劯	7ls=
砡	7lw=
硎	7l0=
硤	7l4=
硺	7l8=
礰	7mA=
礼	7mE=
神	7mI=
祥	7mM=
禔	7mQ=
福	7mU=
禛	7mY=
竑	7mc=
竧	7mg=
靖	7mk=
竫	7mo=
箞	7ms=
精	7mw=
絈	7m0=
絜	7m4=
綷	7m8=
綠	7nA=
緖	7nE=
繒	7nI=
罇	7nM=
羡	7nQ=
羽	7nU=
茁	7nY=
荢	7nc=
荿	7ng=
菇	7nk=
菶	7no=
葈	7ns=
蒴	7nw=
蕓	7n0=
蕙	7n4=
蕫	7oA=
﨟	7oE=
薰	7oI=
蘒	7oM=
﨡	7oQ=
蠇	7oU=
裵	7oY=
訒	7oc=
訷	7og=
詹	7ok=
誧	7oo=
誾	7os=
諟	7ow=
諸	7o0=
諶	7o4=
譓	7o8=
譿	7pA=
賰	7pE=
賴	7pI=
贒	7pM=
赶	7pQ=
﨣	7pU=
軏	7pY=
﨤	7pc=
逸	7pg=
遧	7pk=
郞	7po=
都	7ps=
鄕	7pw=
鄧	7p0=
釚	7p4=
釗	7p8=
釞	7qA=
釭	7qE=
釮	7qI=
釤	7qM=
釥	7qQ=
鈆	7qU=
鈐	7qY=
鈊	7qc=
鈺	7qg=
鉀	7qk=
鈼	7qo=
鉎	7qs=
鉙	7qw=
鉑	7q0=
鈹	7q4=
鉧	7q8=
銧	7rA=
鉷	7rE=
鉸	7rI=
鋧	7rM=
鋗	7rQ=
鋙	7rU=
鋐	7rY=
﨧	7rc=
鋕	7rg=
鋠	7rk=
鋓	7ro=
錥	7rs=
錡	7rw=
鋻	7r0=
﨨	7r4=
錞	7r8=
鋿	7sA=
錝	7sE=
錂	7sI=
鍰	7sM=
鍗	7sQ=
鎤	7sU=
鏆	7sY=
鏞	7sc=
鏸	7sg=
鐱	7sk=
鑅	7so=
鑈	7ss=
閒	7sw=
隆	7s0=
﨩	7s4=
隝	7s8=
隯	7tA=
霳	7tE=
霻	7tI=
靃	7tM=
靍	7tQ=
靏	7tU=
靑	7tY=
靕	7tc=
顗	7tg=
顥	7tk=
飯	7to=
飼	7ts=
餧	7tw=
館	7t0=
馞	7t4=
驎	7t8=
髙	7uA=
髜	7uE=
魵	7uI=
魲	7uM=
鮏	7uQ=
鮱	7uU=
鮻	7uY=
鰀	7uc=
鵰	7ug=
鵫	7uk=
鶴	7uo=
鸙	7us=
黑	7uw=

data = [
    {"col_1": "①"},
    {"col_1": "②"},
    {"col_1": "③"},
    {"col_1": "④"},
    {"col_1": "⑤"},
    {"col_1": "⑥"},
    {"col_1": "⑦"},
    {"col_1": "⑧"},
    {"col_1": "⑨"},
    {"col_1": "⑩"},
    {"col_1": "⑪"},
    {"col_1": "⑫"},
    {"col_1": "⑬"},
    {"col_1": "⑭"},
    {"col_1": "⑮"},
    {"col_1": "⑯"},
    {"col_1": "⑰"},
    {"col_1": "⑱"},
    {"col_1": "⑲"},
    {"col_1": "⑳"},
    {"col_1": "Ⅰ"},
    {"col_1": "Ⅱ"},
    {"col_1": "Ⅲ"},
    {"col_1": "Ⅳ"},
    {"col_1": "Ⅴ"},
    {"col_1": "Ⅵ"},
    {"col_1": "Ⅶ"},
    {"col_1": "Ⅷ"},
    {"col_1": "Ⅸ"},
    {"col_1": "Ⅹ"},
    {"col_1": "ⅰ"},
    {"col_1": "ⅱ"},
    {"col_1": "ⅲ"},
    {"col_1": "ⅳ"},
    {"col_1": "ⅴ"},
    {"col_1": "ⅵ"},
    {"col_1": "ⅶ"},
    {"col_1": "ⅷ"},
    {"col_1": "ⅸ"},
    {"col_1": "ⅹ"},
    {"col_1": "・"},
    {"col_1": "㍉"},
    {"col_1": "㌔"},
    {"col_1": "㌢"},
    {"col_1": "㍍"},
    {"col_1": "㌘"},
    {"col_1": "㌧"},
    {"col_1": "㌃"},
    {"col_1": "㌶"},
    {"col_1": "㍑"},
    {"col_1": "㍗"},
    {"col_1": "㌍"},
    {"col_1": "㌦"},
    {"col_1": "㌣"},
    {"col_1": "㌫"},
    {"col_1": "㍊"},
    {"col_1": "㌻"},
    {"col_1": "㎜"},
    {"col_1": "㎝"},
    {"col_1": "㎞"},
    {"col_1": "㎎"},
    {"col_1": "㎏"},
    {"col_1": "㏄"},
    {"col_1": "㎡"},
    {"col_1": "㍻"},
    {"col_1": "〝"},
    {"col_1": "〟"},
    {"col_1": "№"},
    {"col_1": "㏍"},
    {"col_1": "℡"},
    {"col_1": "㊤"},
    {"col_1": "㊥"},
    {"col_1": "㊦"},
    {"col_1": "㊧"},
    {"col_1": "㊨"},
    {"col_1": "㈱"},
    {"col_1": "㈲"},
    {"col_1": "㈹"},
    {"col_1": "㍾"},
    {"col_1": "㍽"},
    {"col_1": "㍼"},
    {"col_1": "≒"},
    {"col_1": "≡"},
    {"col_1": "∫"},
    {"col_1": "∮"},
    {"col_1": "∑"},
    {"col_1": "√"},
    {"col_1": "⊥"},
    {"col_1": "∠"},
    {"col_1": "⊿"},
    {"col_1": "∵"},
    {"col_1": "∩"},
    {"col_1": "∪"},
    {"col_1": "亜"},
    {"col_1": "唖"},
    {"col_1": "娃"},
    {"col_1": "阿"},
    {"col_1": "哀"},
    {"col_1": "愛"},
    {"col_1": "挨"},
    {"col_1": "姶"},
    {"col_1": "逢"},
    {"col_1": "葵"},
    {"col_1": "茜"},
    {"col_1": "穐"},
    {"col_1": "悪"},
    {"col_1": "握"},
    {"col_1": "渥"},
    {"col_1": "旭"},
    {"col_1": "葦"},
    {"col_1": "￢"},
    {"col_1": "￤"},
    {"col_1": "＇"},
    {"col_1": "＂"},
    {"col_1": "㈱"},
    {"col_1": "№"},
    {"col_1": "℡"},
    {"col_1": "∵"},
    {"col_1": "纊"},
    {"col_1": "褜"},
    {"col_1": "鍈"},
    {"col_1": "銈"},
    {"col_1": "蓜"},
    {"col_1": "俉"},
    {"col_1": "炻"},
    {"col_1": "昱"},
    {"col_1": "棈"},
    {"col_1": "鋹"},
    {"col_1": "曻"},
    {"col_1": "彅"},
    {"col_1": "丨"},
    {"col_1": "仡"},
    {"col_1": "仼"},
    {"col_1": "伀"},
    {"col_1": "伃"},
    {"col_1": "伹"},
    {"col_1": "佖"},
    {"col_1": "侒"},
    {"col_1": "侊"},
    {"col_1": "侚"},
    {"col_1": "侔"},
    {"col_1": "俍"},
    {"col_1": "偀"},
    {"col_1": "倢"},
    {"col_1": "俿"},
    {"col_1": "倞"},
    {"col_1": "偆"},
    {"col_1": "偰"},
    {"col_1": "偂"},
    {"col_1": "傔"},
    {"col_1": "僴"},
    {"col_1": "僘"},
    {"col_1": "兊"},
    {"col_1": "兤"},
    {"col_1": "冝"},
    {"col_1": "冾"},
    {"col_1": "凬"},
    {"col_1": "刕"},
    {"col_1": "劜"},
    {"col_1": "劦"},
    {"col_1": "勀"},
    {"col_1": "勛"},
    {"col_1": "匀"},
    {"col_1": "匇"},
    {"col_1": "匤"},
    {"col_1": "卲"},
    {"col_1": "厓"},
    {"col_1": "厲"},
    {"col_1": "叝"},
    {"col_1": "﨎"},
    {"col_1": "咜"},
    {"col_1": "咊"},
    {"col_1": "咩"},
    {"col_1": "哿"},
    {"col_1": "喆"},
    {"col_1": "坙"},
    {"col_1": "坥"},
    {"col_1": "垬"},
    {"col_1": "埈"},
    {"col_1": "埇"},
    {"col_1": "﨏"},
    {"col_1": "塚"},
    {"col_1": "增"},
    {"col_1": "墲"},
    {"col_1": "夋"},
    {"col_1": "奓"},
    {"col_1": "奛"},
    {"col_1": "奝"},
    {"col_1": "奣"},
    {"col_1": "妤"},
    {"col_1": "妺"},
    {"col_1": "孖"},
    {"col_1": "寀"},
    {"col_1": "甯"},
    {"col_1": "寘"},
    {"col_1": "寬"},
    {"col_1": "尞"},
    {"col_1": "岦"},
    {"col_1": "岺"},
    {"col_1": "峵"},
    {"col_1": "崧"},
    {"col_1": "嵓"},
    {"col_1": "﨑"},
    {"col_1": "嵂"},
    {"col_1": "嵭"},
    {"col_1": "嶸"},
    {"col_1": "嶹"},
    {"col_1": "巐"},
    {"col_1": "弡"},
    {"col_1": "弴"},
    {"col_1": "彧"},
    {"col_1": "德"},
    {"col_1": "忞"},
    {"col_1": "恝"},
    {"col_1": "悅"},
    {"col_1": "悊"},
    {"col_1": "惞"},
    {"col_1": "惕"},
    {"col_1": "愠"},
    {"col_1": "惲"},
    {"col_1": "愑"},
    {"col_1": "愷"},
    {"col_1": "愰"},
    {"col_1": "憘"},
    {"col_1": "戓"},
    {"col_1": "抦"},
    {"col_1": "揵"},
    {"col_1": "摠"},
    {"col_1": "撝"},
    {"col_1": "擎"},
    {"col_1": "敎"},
    {"col_1": "昀"},
    {"col_1": "昕"},
    {"col_1": "昻"},
    {"col_1": "昉"},
    {"col_1": "昮"},
    {"col_1": "昞"},
    {"col_1": "昤"},
    {"col_1": "晥"},
    {"col_1": "晗"},
    {"col_1": "晙"},
    {"col_1": "晴"},
    {"col_1": "晳"},
    {"col_1": "暙"},
    {"col_1": "暠"},
    {"col_1": "暲"},
    {"col_1": "暿"},
    {"col_1": "曺"},
    {"col_1": "朎"},
    {"col_1": "朗"},
    {"col_1": "杦"},
    {"col_1": "枻"},
    {"col_1": "桒"},
    {"col_1": "柀"},
    {"col_1": "栁"},
    {"col_1": "桄"},
    {"col_1": "棏"},
    {"col_1": "﨓"},
    {"col_1": "楨"},
    {"col_1": "﨔"},
    {"col_1": "榘"},
    {"col_1": "槢"},
    {"col_1": "樰"},
    {"col_1": "橫"},
    {"col_1": "橆"},
    {"col_1": "橳"},
    {"col_1": "橾"},
    {"col_1": "櫢"},
    {"col_1": "櫤"},
    {"col_1": "毖"},
    {"col_1": "氿"},
    {"col_1": "汜"},
    {"col_1": "沆"},
    {"col_1": "汯"},
    {"col_1": "泚"},
    {"col_1": "洄"},
    {"col_1": "涇"},
    {"col_1": "浯"},
    {"col_1": "涖"},
    {"col_1": "涬"},
    {"col_1": "淏"},
    {"col_1": "淸"},
    {"col_1": "淲"},
    {"col_1": "淼"},
    {"col_1": "渹"},
    {"col_1": "湜"},
    {"col_1": "渧"},
    {"col_1": "渼"},
    {"col_1": "溿"},
    {"col_1": "澈"},
    {"col_1": "澵"},
    {"col_1": "濵"},
    {"col_1": "瀅"},
    {"col_1": "瀇"},
    {"col_1": "瀨"},
    {"col_1": "炅"},
    {"col_1": "炫"},
    {"col_1": "焏"},
    {"col_1": "焄"},
    {"col_1": "煜"},
    {"col_1": "煆"},
    {"col_1": "煇"},
    {"col_1": "凞"},
    {"col_1": "燁"},
    {"col_1": "燾"},
    {"col_1": "犱"},
    {"col_1": "犾"},
    {"col_1": "猤"},
    {"col_1": "猪"},
    {"col_1": "獷"},
    {"col_1": "玽"},
    {"col_1": "珉"},
    {"col_1": "珖"},
    {"col_1": "珣"},
    {"col_1": "珒"},
    {"col_1": "琇"},
    {"col_1": "珵"},
    {"col_1": "琦"},
    {"col_1": "琪"},
    {"col_1": "琩"},
    {"col_1": "琮"},
    {"col_1": "瑢"},
    {"col_1": "璉"},
    {"col_1": "璟"},
    {"col_1": "甁"},
    {"col_1": "畯"},
    {"col_1": "皂"},
    {"col_1": "皜"},
    {"col_1": "皞"},
    {"col_1": "皛"},
    {"col_1": "皦"},
    {"col_1": "益"},
    {"col_1": "睆"},
    {"col_1": "劯"},
    {"col_1": "砡"},
    {"col_1": "硎"},
    {"col_1": "硤"},
    {"col_1": "硺"},
    {"col_1": "礰"},
    {"col_1": "礼"},
    {"col_1": "神"},
    {"col_1": "祥"},
    {"col_1": "禔"},
    {"col_1": "福"},
    {"col_1": "禛"},
    {"col_1": "竑"},
    {"col_1": "竧"},
    {"col_1": "靖"},
    {"col_1": "竫"},
    {"col_1": "箞"},
    {"col_1": "精"},
    {"col_1": "絈"},
    {"col_1": "絜"},
    {"col_1": "綷"},
    {"col_1": "綠"},
    {"col_1": "緖"},
    {"col_1": "繒"},
    {"col_1": "罇"},
    {"col_1": "羡"},
    {"col_1": "羽"},
    {"col_1": "茁"},
    {"col_1": "荢"},
    {"col_1": "荿"},
    {"col_1": "菇"},
    {"col_1": "菶"},
    {"col_1": "葈"},
    {"col_1": "蒴"},
    {"col_1": "蕓"},
    {"col_1": "蕙"},
    {"col_1": "蕫"},
    {"col_1": "﨟"},
    {"col_1": "薰"},
    {"col_1": "蘒"},
    {"col_1": "﨡"},
    {"col_1": "蠇"},
    {"col_1": "裵"},
    {"col_1": "訒"},
    {"col_1": "訷"},
    {"col_1": "詹"},
    {"col_1": "誧"},
    {"col_1": "誾"},
    {"col_1": "諟"},
    {"col_1": "諸"},
    {"col_1": "諶"},
    {"col_1": "譓"},
    {"col_1": "譿"},
    {"col_1": "賰"},
    {"col_1": "賴"},
    {"col_1": "贒"},
    {"col_1": "赶"},
    {"col_1": "﨣"},
    {"col_1": "軏"},
    {"col_1": "﨤"},
    {"col_1": "逸"},
    {"col_1": "遧"},
    {"col_1": "郞"},
    {"col_1": "都"},
    {"col_1": "鄕"},
    {"col_1": "鄧"},
    {"col_1": "釚"},
    {"col_1": "釗"},
    {"col_1": "釞"},
    {"col_1": "釭"},
    {"col_1": "釮"},
    {"col_1": "釤"},
    {"col_1": "釥"},
    {"col_1": "鈆"},
    {"col_1": "鈐"},
    {"col_1": "鈊"},
    {"col_1": "鈺"},
    {"col_1": "鉀"},
    {"col_1": "鈼"},
    {"col_1": "鉎"},
    {"col_1": "鉙"},
    {"col_1": "鉑"},
    {"col_1": "鈹"},
    {"col_1": "鉧"},
    {"col_1": "銧"},
    {"col_1": "鉷"},
    {"col_1": "鉸"},
    {"col_1": "鋧"},
    {"col_1": "鋗"},
    {"col_1": "鋙"},
    {"col_1": "鋐"},
    {"col_1": "﨧"},
    {"col_1": "鋕"},
    {"col_1": "鋠"},
    {"col_1": "鋓"},
    {"col_1": "錥"},
    {"col_1": "錡"},
    {"col_1": "鋻"},
    {"col_1": "﨨"},
    {"col_1": "錞"},
    {"col_1": "鋿"},
    {"col_1": "錝"},
    {"col_1": "錂"},
    {"col_1": "鍰"},
    {"col_1": "鍗"},
    {"col_1": "鎤"},
    {"col_1": "鏆"},
    {"col_1": "鏞"},
    {"col_1": "鏸"},
    {"col_1": "鐱"},
    {"col_1": "鑅"},
    {"col_1": "鑈"},
    {"col_1": "閒"},
    {"col_1": "隆"},
    {"col_1": "﨩"},
    {"col_1": "隝"},
    {"col_1": "隯"},
    {"col_1": "霳"},
    {"col_1": "霻"},
    {"col_1": "靃"},
    {"col_1": "靍"},
    {"col_1": "靏"},
    {"col_1": "靑"},
    {"col_1": "靕"},
    {"col_1": "顗"},
    {"col_1": "顥"},
    {"col_1": "飯"},
    {"col_1": "飼"},
    {"col_1": "餧"},
    {"col_1": "館"},
    {"col_1": "馞"},
    {"col_1": "驎"},
    {"col_1": "髙"},
    {"col_1": "髜"},
    {"col_1": "魵"},
    {"col_1": "魲"},
    {"col_1": "鮏"},
    {"col_1": "鮱"},
    {"col_1": "鮻"},
    {"col_1": "鰀"},
    {"col_1": "鵰"},
    {"col_1": "鵫"},
    {"col_1": "鶴"},
    {"col_1": "鸙"},
    {"col_1": "黑"},
]

df = pd.DataFrame(data)
df['col_1_encoded'] = df['col_1'].str.encode('cp932')
df