0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?

More than 5 years have passed since last update.

A template of a C++ parser with RE/flex and Bison

0
Last updated at Posted at 2020-09-20

なぜ, 最近DXRの記事が見られているの?今更やろ. RTX3080? Siggraph? CEDEC?

はじめに

通信パケットの定義など, ちょっとしたパーサが必要になることが多いです.
Protocol Buffersなどが使えればよいのですが.
将来の手間を減らすために, テンプレートを作っておきます.

環境

RE/flex

Unicode対応を謳うFlex互換を見つけたので, これでいきましょう. 検証はしてない, 私はASCII以外使わないので.
RE/flexとドキュメント.

WinFlexBison - Flex and Bison for Microsoft Windows

Linuxならパッケージがあるでしょう. 他人がメンテナンスできる必要があるため, 素のWindowsで動かせるようにします. Bisonだけとりだします.
WinFlexBison

単語分割

空白で単語を分割してみます.

words.l
%top{
    #include "Parser.hpp"
}

%option noyywrap
%option unicode
%option header-file
%option bison-cc
%option bison-complete
%option bison-cc-namespace=yy
%option bison-cc-parser=Parser
%option bison-locations

%option exception="yy::Parser::syntax_error(location(), \"Unknown token.\")"

%option namespace=yy
%option lexer=Lexer
%option c++

%class{
public:
    void init();
}

newline \r?\n
//in unicode mode, \s includes Unicode sub-property Zs
spaces \s+
//in unicode mode, \w includes Unicode word characters with property L, Nd, or Pc  
word	\w+

%%
{spaces}    // skip spaces
{word}      return yy::Parser::make_WORD(str(), location());
<<EOF>>     return yy::Parser::make_EOF(location());
%%

namespace yy
{
    void Lexer::init()
    {
        std::cout << "Lexer::init" << std::endl;
    }
}
words.yy
%skeleton "lalr1.cc"

%require "3.2"
%language "c++"

%define api.namespace {yy}
%define api.parser.class {Parser}
%define api.value.type variant
%define api.token.constructor

%define parse.error verbose

%defines
%output "Parser.cpp"

%locations
%define api.location.file "Location.h"

%code requires{
    namespace yy {
        class Lexer;
    }
}

%parse-param { yy::Lexer& lexer }

%code {
    #include "lex.yy.h" // a header file generated with reflex --header-file
    #undef yylex
    #define yylex lexer.lex //replace calling the global yylex() with calling the lexer.lex()

    namespace yy{
    std::ostream& operator<<(std::ostream& os, const std::vector<std::string>& ss)
    {
        for(const std::string& s : ss){
            os << s << ' ';
        }
        return os;
    }
    }
}

%define api.token.prefix {TOK_}
%token <std::string> WORD "word"
%token EOF 0 "end of file"
%nterm <std::vector<std::string>> list;
%nterm <std::string> item;

%%
result:
    list    {std::cout << $1 << std::endl;}
;
list:
    %empty  {}
| list item { $$ = $1; $$.push_back($2);}
;
item:
    WORD    { $$ = $1;}
%%

void yy::Parser::error(const location_type& l, const std::string& msg)
{
    std::cerr << l << ": " << msg << std::endl;
}
main.cpp
# include "Parser.hpp"
# include "lex.yy.h"

int main(void)
{
    std::string str("aaa \nbbb \nccc \nddd");
    yy::Lexer lexer;
    lexer.init();
    lexer.in(str);
    yy::Parser parser(lexer);
    return parser.parse();
}

batchなりshellなりで生成できるようにしておきます.

$ .\flex\reflex words.l
$ .\bison\bison words.yy

パケット定義テンプレート

ハゲオヤジ追記中(2020//09/20)
Protocol Buffersを参考に1分で考えたテンプレート, 運用しながらテンプレートも改良しましょう.

まとめ

ドキュメントが少ないのは, この手のツールあるあるですね. 結局理解できる人しか使わないので, 私のような無能には導入でも難しい.

0
0
0

Register as a new user and use Qiita more conveniently

  1. You get articles that match your needs
  2. You can efficiently read back useful information
  3. You can use dark theme
What you can do with signing up
0
0

Delete article

Deleted articles cannot be recovered.

Draft of this article would be also deleted.

Are you sure you want to delete this article?