Help us understand the problem. What is going on with this article?

Goodby/CSV 大量にデータをインポートすると、マルチバイト文字列が欠落する場合がある

自分用の備忘録

何が問題か

私が教えてほしいです。

大量に(400行ぐらい)データをインポートすると、1~数行、
日本語文字列が一文字(3バイト分)だけ欠落するという事象が発生した

原因は不明。誰か教えて

直し方

これを

$config->setFromCharset("UTF-8");

こうすれば直った

$config->setFromCharset(NULL);

ソースコード

    /**
     * Import Csv By Goodby\CSV Parser
     * このメソッドはCSVの読み込みのみを担当し、加工はコントローラー側に任せる想定
     *
     * @param  string $filePath
     * @return array
     * @link https://github.com/goodby/csv
     * @link https://nori-life.com/laravel-csv-import-goodbycsv/
     */
    public function import($filePath)
    {
        // variable definition
        $rows = array();

        // goodby csv configuration
        $config = new LexerConfig();
        $config->setDelimiter(","); // e.g. [",", "\t"]
        $config->setEnclosure('"'); // e.g. ["'", '"']
        $config->setEscape("\\");

        // UTF-8にすると、大量にデータを読み込んだ際に、マルチバイト文字列が欠落する場合がある模様
        //$config->setFromCharset("UTF-8"); // e.g. [NULL, "UTF-8", "SJIS-win", "EUC-JP"]
        $config->setFromCharset(NULL); // e.g. [NULL, "UTF-8", "SJIS-win", "EUC-JP"]

        $config->setToCharset("UTF-8"); // e.g. [NULL, "UTF-8", "SJIS-win", "EUC-JP"]
        $config->setIgnoreHeaderLine(TRUE);

        // format csv
        $interpreter = new Interpreter();
        $interpreter->addObserver(function(array $row) use (&$rows){
            $rows[] = $row;
        });

        // parse csv
        $lexer = new Lexer($config);
        $lexer->parse($filePath, $interpreter); // parsed csv data into $rows here.

        return $rows;
    }
Why not register and get more from Qiita?
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
Comments
No comments
Sign up for free and join this conversation.
If you already have a Qiita account
Why do not you register as a user and use Qiita more conveniently?
You need to log in to use this function. Qiita can be used more conveniently after logging in.
You seem to be reading articles frequently this month. Qiita can be used more conveniently after logging in.
  1. We will deliver articles that match you
    By following users and tags, you can catch up information on technical fields that you are interested in as a whole
  2. you can read useful information later efficiently
    By "stocking" the articles you like, you can search right away
ユーザーは見つかりませんでした