LoginSignup
0
0

More than 5 years have passed since last update.

file命令所用magic数据

Posted at

Q:
$ file -b S1650V100R005.lzma
LZMA compressed data, non-streamed, size 3173380

大概在2011年的时候,file命令还能正确识别一个经LZMA压缩过的文件,2015年在一个不同的环境中:
$ file -b S1650V100R005.lzma
data

目标文件是同一个,怎么会这样?

A: scz 2015-01-29 11:12

file只是一个文件类型识别框架,用于识别文件类型的特征数据存放在:

/usr/share/misc/magic/ 该目录下存放可读的magic文件
/usr/share/misc/magic.mgc 可读magic文件的预编译版本(二进制)

/usr/share/file/magic/
/usr/share/file/magic.mgc

关于可读magic文件的格式,参看magic(5)。
$ dpkg -S which file
file: /usr/bin/file
$ dpkg -L file
$ dpkg -S /usr/share/file/magic.mgc
libmagic1: /usr/share/file/magic.mgc
$ dpkg -L libmagic1

Debian、Ubuntu的libmagic1包不知出于什么考虑,只提供了预编译的magic.mgc。如果想看可读版本,只能自己下载file命令的源码包,其中包含可读magic。假设sources.list中已经有deb-src条目,可以这样下载file源码:
cd /tmp
mkdir src
cd src
apt-get source file
dpkg-source -x file_5.11-2+deb7u6.dsc (好像没必要,已经是打过补丁的状态?)
cd file-5.11/magic/Magdir
grep -i -Hn lzma *

compress:192:# Type: LZMA
compress:194:>12 leshort =0xff LZMA compressed data,
compress:197:!:mime application/x-lzma
filesystems:1357:>31 byte 3 (lzma),

2011年能工作的file是5.04版,"apt-get source file"有点问题,直接下载源码包:

cd file-5.04/magic/Magdir
grep -i -Hn lzma *

compress:192:# Type: LZMA
compress:197:#0 string ]\000\000\200\000 LZMA compressed data
filesystems:1343:>31 byte 3 (lzma),

实测表明5.04版的可读magic同样不能识别S1650V100R005.lzma。后来我想起来了,2011年的时候我从binwalk中复制了一些magic数据补充到/etc/magic中,这才是导致区别的根本原因。官方magic数据直至今日从未成功识别S1650V100R005.lzma。

观察file命令以何种顺序获取magic数据:

$ strace -efile file -m /tmp/anything -b which ls 2>&1 | grep -E "/magic|/anything"

/tmp/anything.mgc
/tmp/anything

$ strace -efile file -b which ls 2>&1 | grep -E "/magic|/anything"

/etc/magic.mgc
/etc/magic
/usr/share/misc/magic.mgc
/usr/share/misc/magic/

magic数据的预编译版本优先级高于可读版本。指定-m时,不再读取系统缺省magic数据。-m先尝试anything.mgc,再尝试anything。假设anything等于some.mgc,会先寻找some.mgc.mgc,肯定失败,接着找到some.mgc,但这次是把some.mgc当成可读版本处理,于是你会发现终端显示进入黑方块状态,必须用Ctrl-V/O技巧恢复。我估计没几个人正经用过-m,file的man手册关于-m的说明晦涩不明。

现在回到原始问题,解决办法是,先获取当前/usr/share/misc/magic.mgc对应的可读版本,再从binwalk中补充一些额外的magic数据。

$ ls -l /usr/local/lib/python2.7/dist-packages/binwalk/magic
-rw-r--r-- 1 root staff 14514 Dec 31 15:18 archives
-rw-r--r-- 1 root staff 3597 Dec 31 15:18 binarch
-rw-r--r-- 1 root staff 176 Dec 31 15:18 bincast
-rw-r--r-- 1 root staff 176 Dec 31 15:18 binwalk
-rw-r--r-- 1 root staff 386 Dec 31 15:18 bootloaders
-rw-r--r-- 1 root staff 176 Dec 31 15:18 code
-rw-r--r-- 1 root staff 9255 Dec 31 15:18 compressed // x
-rw-r--r-- 1 root staff 5372 Dec 31 15:18 console
-rw-r--r-- 1 root staff 4929 Dec 31 15:18 crypto
-rw-r--r-- 1 root staff 2194 Dec 31 15:18 ecos
-rw-r--r-- 1 root staff 237 Dec 31 15:18 encoding // x
-rw-r--r-- 1 root staff 26969 Dec 31 15:18 executables
-rw-r--r-- 1 root staff 25830 Dec 31 15:18 filesystems // x
-rw-r--r-- 1 root staff 27397 Dec 31 15:18 firmware // x
-rw-r--r-- 1 root staff 512 Dec 31 15:18 hashing
-rw-r--r-- 1 root staff 11544 Dec 31 15:18 images
-rw-r--r-- 1 root staff 880 Dec 31 15:18 linux
-rw-r--r-- 1 root staff 108749 Dec 31 15:18 lzma
-rw-r--r-- 1 root staff 3119 Dec 31 15:18 misc // x
-rw-r--r-- 1 root staff 9017 Dec 31 15:18 network
-rw-r--r-- 1 root staff 2397 Dec 31 15:18 sql
-rw-r--r-- 1 root staff 9986 Dec 31 15:18 vxworks
$ cd /usr/local/lib/python2.7/dist-packages/binwalk/magic
$ cat archives binarch bincast binwalk bootloaders code console crypto ecos executables hashing images linux lzma network sql vxworks > /tmp/magic.binwalk

上面标x的magic数据与file命令一起使用时会报错。

$ file -m /tmp/magic.binwalk -b which ls
ELF, 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.26, BuildID[sha1]=0xd3280633faaabf56a14a26693d2f810a32222e51, stripped
$ file -m /tmp/magic.binwalk -b S1650V100R005.lzma
LZMA compressed data, properties: 0x5D, dictionary size: 8388608 bytes, uncompressed size: 3173380 bytes

$ ls -l /tmp/src/file-5.11/magic/Magdir/*
$ cat /tmp/src/file-5.11/magic/Magdir/* > /tmp/magic.file-5.11
$ file -m /tmp/magic.file-5.11 -b which ls
ELF 32-bit LSB executable, Intel 80386, version 1 (SYSV), dynamically linked (uses shared libs), for GNU/Linux 2.6.26, BuildID[sha1]=0xd3280633faaabf56a14a26693d2f810a32222e51, stripped
$ file -m /tmp/magic.file-5.11 -b /root/src/H3C/S1650V100R005.lzma
data

这两份magic数据可以合并:

$ cat /tmp/magic.binwalk /tmp/magic.file-5.11 > /tmp/magic.all
$ ls -l /tmp/magic.all
-rw-r--r-- 1 root root 823052 Jan 29 14:15 /tmp/magic.all
$ cd /usr/share/file/
$ file -C -m /tmp/magic.all
$ ls -l magic.all.mgc
-rw-r--r-- 1 root root 2986768 Jan 29 14:16 magic.all.mgc
$ mv magic.all.mgc magic.mgc

$ file -b S1650V100R005.lzma
LZMA compressed data, properties: 0x5D, dictionary size: 8388608 bytes, uncompressed size: 3173380 bytes

现在是不是觉得自己的file瞬间高大上了?遗憾的是,没有找到现成的将magic.mgc转换成magic的工具。

$ file -b magic.mgc
magic binary file for file(1) cmd (version 8) (little endian)

附件是magic.all与magic.mgc,替换/usr/share/file/magic.mgc即可使用增强版magic数据。

0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
0