lxmldump: lxmldump.py annotate

annotate lxmldump.py @ 56:71e66eca1e23

Merge.

author	Matti Hamalainen <ccr@tnsp.org>
date	Thu, 27 May 2021 16:55:22 +0300
parents	301452a71cc7 884770576e74
children	4410e2d1bff3

rev	line source
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	1 #!/usr/bin/python3 -B
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	2 # coding=utf-8
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	3 ###
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	4 ### lxmldump - Convert and dump ISO/FDIS 1951 XML file data
4 60b789dfee32 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 1 diff changeset	5 ### Programmed and designed by Matti 'ccr' Hämäläinen <ccr@tnsp.org>
60b789dfee32 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 1 diff changeset	6 ### (C) Copyright 2021 Tecnic Software productions (TNSP)
60b789dfee32 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 1 diff changeset	7 ###
23 3ef1c5463b8f Add license. Matti Hamalainen <ccr@tnsp.org> parents: 22 diff changeset	8 ### Released / distributed under 3-clause BSD license
3ef1c5463b8f Add license. Matti Hamalainen <ccr@tnsp.org> parents: 22 diff changeset	9 ### (see file "COPYING" for more information)
3ef1c5463b8f Add license. Matti Hamalainen <ccr@tnsp.org> parents: 22 diff changeset	10 ###
4 60b789dfee32 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 1 diff changeset	11 ### Python 3.7+ required!
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	12 ###
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	13 import sys
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	14 import signal
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	15 import re
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	16 from pathlib import Path
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	17 import xml.etree.ElementTree as xmlET
5 274b2091137c Some more work on cleaning this up. Matti Hamalainen <ccr@tnsp.org> parents: 4 diff changeset	18 import unicodedata
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	19 import argparse
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	20 import textwrap
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	21
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	22 assert sys.version_info >= (3, 7)
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	23
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	24
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	25 ###
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	26 ### Default settings
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	27 ###
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	28 # Operation modes
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	29 PKK_MODE_NORMAL = 0
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	30 PKK_MODE_DUMP = 1
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	31 PKK_MODE_XML = 2
35 5aafa87dbec2 Some Anki-related stuff, not finalized and thus hidden. Matti Hamalainen <ccr@tnsp.org> parents: 34 diff changeset	32 PKK_MODE_ANKI = 3
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	33
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	34
30 34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	35 pkk_modes_list = {
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	36 PKK_MODE_NORMAL: "normal",
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	37 PKK_MODE_DUMP: "dump",
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	38 PKK_MODE_XML: "xml",
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	39 PKK_MODE_ANKI: "anki",
30 34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	40 }
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	41
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	42
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	43 pkk_mode_defaults = {
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	44 # Default Ptr URL format strings
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	45 "ptr_fmt": {
52 95671cdda422 Remove u specifier. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	46 PKK_MODE_NORMAL: "<PTR:{href}>{text}</PTR>",
95671cdda422 Remove u specifier. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	47 PKK_MODE_ANKI: "<a href='https://kaino.kotus.fi/cgi-bin/kks/karjala.cgi?a={href}'>{text}</a>",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	48 },
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	49
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	50 "word_item": {
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	51 PKK_MODE_NORMAL: "\"{word}\"{search}{attr}\n{hyphenation}{main_sense}{other_senses}\n",
51 5d3fb2f3aa21 Remove cleanup function as it is unused. Matti Hamalainen <ccr@tnsp.org> parents: 50 diff changeset	52 PKK_MODE_ANKI: "\"{word}\"{search}{attr}{hyphenation};{main_sense};{other_senses}\n",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	53 },
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	54 "word_attr_list": {
bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	55 PKK_MODE_NORMAL: " ({alist}) ",
49 d3d4b547f86c Some prodding at Anki output. Matti Hamalainen <ccr@tnsp.org> parents: 48 diff changeset	56 PKK_MODE_NORMAL: " ({alist})",
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	57 },
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	58 "word_attr_list_empty": {
bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	59 PKK_MODE_NORMAL: " ",
49 d3d4b547f86c Some prodding at Anki output. Matti Hamalainen <ccr@tnsp.org> parents: 48 diff changeset	60 PKK_MODE_ANKI: "",
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	61 },
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	62 "word_attr_list_item": {
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	63 PKK_MODE_NORMAL: "{text}",
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	64 },
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	65 "word_attr_list_sep": {
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	66 PKK_MODE_NORMAL: " ; ",
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	67 PKK_MODE_ANKI: " : ",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	68 },
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	69
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	70 "search_list": {
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	71 PKK_MODE_NORMAL: ", {alist}",
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	72 },
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	73 "search_list_empty": {
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	74 PKK_MODE_NORMAL: "",
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	75 },
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	76 "search_list_item": {
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	77 PKK_MODE_NORMAL: "\"{text}\"",
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	78 },
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	79 "search_list_sep": {
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	80 PKK_MODE_NORMAL: ", ",
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	81 },
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	82
44 d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	83 "hyphenation": {
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	84 PKK_MODE_NORMAL: "{indent}hyph \"{text}\"\n",
50 5b22ffdab0ce Adjustment. Matti Hamalainen <ccr@tnsp.org> parents: 49 diff changeset	85 PKK_MODE_ANKI: " [hyph: {text}]",
44 d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	86 },
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	87 "no_hyphenation": {
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	88 PKK_MODE_NORMAL: "",
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	89 },
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	90
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	91 "sense_list": {
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	92 PKK_MODE_NORMAL: "{alist}",
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	93 },
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	94 "sense_list_empty": {
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	95 PKK_MODE_NORMAL: "",
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	96 },
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	97 "sense_list_item": {
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	98 PKK_MODE_NORMAL: "{indent}sense #{index}\n{text}",
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	99 PKK_MODE_ANKI: "{text}",
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	100 },
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	101 "sense_list_sep": {
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	102 PKK_MODE_NORMAL: "",
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	103 PKK_MODE_NORMAL: ";",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	104 },
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	105
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	106 "definition_item": {
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	107 PKK_MODE_NORMAL: "{indent}defn \"{text}\"\n",
49 d3d4b547f86c Some prodding at Anki output. Matti Hamalainen <ccr@tnsp.org> parents: 48 diff changeset	108 PKK_MODE_ANKI: " * \"{text}\"",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	109 },
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	110
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	111 "example_item": {
54 884770576e74 Rename a token. Matti Hamalainen <ccr@tnsp.org> parents: 53 diff changeset	112 PKK_MODE_NORMAL: "{indent}exmp \"{text}\"{geo_list}\n",
884770576e74 Rename a token. Matti Hamalainen <ccr@tnsp.org> parents: 53 diff changeset	113 PKK_MODE_ANKI: " ⚫ \"{text}\"{geo_list}",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	114 },
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	115 "example_geo_list": {
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	116 PKK_MODE_NORMAL: " ({alist})",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	117 },
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	118 "example_geo_list_empty": {
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	119 PKK_MODE_NORMAL: "",
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	120 },
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	121 "example_geo_list_item": {
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	122 PKK_MODE_NORMAL: "{text} [{tclass}]",
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	123 },
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	124 "example_geo_list_sep": {
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	125 PKK_MODE_NORMAL: ", ",
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	126 },
30 34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	127 }
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	128
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	129
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	130 # Element annotation mappings
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	131 pkk_element_annotation_map = {
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	132 "Fragment" : {
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	133 PKK_MODE_NORMAL: ["<", ">"],
35 5aafa87dbec2 Some Anki-related stuff, not finalized and thus hidden. Matti Hamalainen <ccr@tnsp.org> parents: 34 diff changeset	134 PKK_MODE_ANKI: ["<", ">"],
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	135 },
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	136 }
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	137
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	138
28 3442b8700da7 Comments. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	139 # List of words in kks1/ useful for debugging, option -p
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	140 pkk_debug_list = [
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	141 "ahas",
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	142 "ahavakkaine",
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	143 "ahavakala",
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	144 "ahavakoittuo",
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	145 "ahvaliha",
9 2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	146 "aloilleh",
18 ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	147 "hanjahtoakseh",
19 7c6eb57798bd Ja niin. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	148 "akkalisto",
33 3dcf8ac43bda Add one word to debug list. Matti Hamalainen <ccr@tnsp.org> parents: 32 diff changeset	149 "alto-",
50 5b22ffdab0ce Adjustment. Matti Hamalainen <ccr@tnsp.org> parents: 49 diff changeset	150 "allot-",
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	151 ]
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	152
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	153
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	154 pkk_settings = {}
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	155
21 7ef08e05a5bf Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	156
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	157 ###
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	158 ### Misc. helper functions, etc
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	159 ###
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	160
13 3bd772fd6a50 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 12 diff changeset	161 ## Print string to stdout using normalized Unicode if enabled
5 274b2091137c Some more work on cleaning this up. Matti Hamalainen <ccr@tnsp.org> parents: 4 diff changeset	162 def pkk_print(smsg):
12 d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	163 try:
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	164 if pkk_cfg.normalize:
12 d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	165 sys.stdout.write(unicodedata.normalize("NFC", smsg))
d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	166 else:
d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	167 sys.stdout.write(smsg)
d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	168
d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	169 except (BrokenPipeError, IOError) as e:
d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	170 sys.stderr.close()
d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	171
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	172
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	173 ## Get indentation string
bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	174 def pkk_geti(indent):
bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	175 return " " * pkk_cfg.indent * indent
bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	176
bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	177
13 3bd772fd6a50 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 12 diff changeset	178 ## Print string with indentation
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	179 def pkk_printi(indent, smsg):
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	180 pkk_print(pkk_geti(indent) + smsg)
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	181
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	182
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	183 ## Fatal error handler
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	184 def pkk_fatal(smsg):
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	185 print(u"ERROR: "+ smsg)
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	186 sys.exit(1)
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	187
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	188
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	189 ## Handler for SIGINT signals
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	190 def pkk_signal_handler(signal, frame):
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	191 print(u"\nQuitting due to SIGINT / Ctrl+C!")
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	192 sys.exit(1)
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	193
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	194
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	195 class pkk_set_mode(argparse.Action):
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	196 def __call__(self, parser, namespace, value, option_string=None):
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	197 lvalue = value.strip().lower()
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	198 for mode in pkk_modes_list:
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	199 if pkk_modes_list[mode] == lvalue:
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	200 setattr(namespace, self.dest, mode)
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	201 return
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	202
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	203 pkk_fatal(f"Invalid output mode '{lvalue}'.")
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	204
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	205
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	206 ## Value handling
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	207 class pkk_set_value(argparse.Action):
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	208
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	209 rexpr = re.compile(r'\s(\w+)\s=\s(.)\s*')
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	210
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	211 def __call__(self, parser, namespace, values, option_string=None):
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	212 rmatch = re.match(self.rexpr, values)
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	213 if rmatch:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	214 rid = rmatch.group(1).lower().replace("-", "_")
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	215 rval = rmatch.group(2)
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	216 if rid in pkk_mode_defaults:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	217 pkk_settings[rid] = rval
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	218 else:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	219 pkk_fatal(f"Invalid option '{option_string} {values}': No such ID '{rid}'.")
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	220 else:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	221 pkk_fatal(f"Invalid option '{option_string} {values}': Expected id=value.")
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	222
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	223
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	224 ## Get mode if it exists
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	225 def pkk_test_value(mid):
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	226 if mid in pkk_mode_defaults:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	227 if pkk_cfg.mode in pkk_mode_defaults[mid]:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	228 mmode = pkk_cfg.mode
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	229 else:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	230 mmode = PKK_MODE_NORMAL
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	231
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	232 if mmode in pkk_mode_defaults[mid]:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	233 return mmode
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	234 else:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	235 return None
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	236 else:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	237 return None
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	238
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	239
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	240 ## Get default value per mode
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	241 def pkk_get_value(mid):
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	242 if mid in pkk_settings and pkk_settings[mid] != None:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	243 return pkk_settings[mid]
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	244
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	245 mmode = pkk_test_value(mid)
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	246 if mmode == None:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	247 pkk_fatal(f"Internal error: No mode for ID '{mid}'.")
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	248
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	249 return pkk_mode_defaults[mid][mmode]
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	250
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	251
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	252 def pkk_get_fmt(mid):
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	253 return pkk_get_value(mid).replace("\\n", "\n")
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	254
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	255
28 3442b8700da7 Comments. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	256 ## Annotate given string with prefix and suffix based on tag
26 420f13925f20 Rename a variable. Matti Hamalainen <ccr@tnsp.org> parents: 25 diff changeset	257 def pkk_str_annotate(mtag, mstr):
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	258 if pkk_cfg.annotate and mtag in pkk_element_annotation_map:
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	259 if pkk_cfg.mode in pkk_element_annotation_map[mtag]:
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	260 mmode = pkk_cfg.mode
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	261 else:
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	262 mmode = PKK_MODE_NORMAL
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	263
26 420f13925f20 Rename a variable. Matti Hamalainen <ccr@tnsp.org> parents: 25 diff changeset	264 return pkk_element_annotation_map[mtag][mmode][0] + mstr + pkk_element_annotation_map[mtag][mmode][1]
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	265 else:
26 420f13925f20 Rename a variable. Matti Hamalainen <ccr@tnsp.org> parents: 25 diff changeset	266 return mstr
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	267
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	268
13 3bd772fd6a50 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 12 diff changeset	269 ## Clean string by removing tabs and newlines
10 013f0cd9e5b3 More work. Matti Hamalainen <ccr@tnsp.org> parents: 9 diff changeset	270 def pkk_str_clean(mstr):
013f0cd9e5b3 More work. Matti Hamalainen <ccr@tnsp.org> parents: 9 diff changeset	271 return re.sub(r'[\n\r\t]', '', mstr)
013f0cd9e5b3 More work. Matti Hamalainen <ccr@tnsp.org> parents: 9 diff changeset	272
013f0cd9e5b3 More work. Matti Hamalainen <ccr@tnsp.org> parents: 9 diff changeset	273
28 3442b8700da7 Comments. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	274 ## Format a "Ptr" node as text
9 2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	275 def pkk_ptr_to_text(pnode):
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	276 pfmt = pkk_get_fmt("ptr_fmt")
30 34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	277 return pfmt.format(
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	278 text=("".join(pnode.itertext())).strip(),
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	279 href=pnode.attrib["{http://www.w3.org/TR/xlink}href"])
9 2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	280
2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	281
13 3bd772fd6a50 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 12 diff changeset	282 ## Get text inside a given node
16 285b0820d2c6 Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 15 diff changeset	283 def pkk_node_to_text(lnode):
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	284 stmp = ""
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	285 for pnode in lnode.iter():
9 2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	286 if pnode.tag == "Ptr":
2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	287 stmp += pkk_ptr_to_text(pnode)
2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	288 else:
2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	289 if isinstance(pnode.text, str):
37 e176fcfc0235 Fix overstripping of concatenated strings. Matti Hamalainen <ccr@tnsp.org> parents: 36 diff changeset	290 stmp += pkk_str_annotate(pnode.tag, pkk_str_clean(pnode.text))
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	291
9 2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	292 if isinstance(pnode.tail, str):
10 013f0cd9e5b3 More work. Matti Hamalainen <ccr@tnsp.org> parents: 9 diff changeset	293 stmp += pkk_str_clean(pnode.tail)
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	294
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	295 return stmp.strip()
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	296
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	297
13 3bd772fd6a50 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 12 diff changeset	298 ## Simple recursive dump starting at given node
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	299 def pkk_dump_recursive(indent, lnode):
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	300 if lnode.tag in ["Example"]:
16 285b0820d2c6 Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 15 diff changeset	301 stmp = pkk_node_to_text(lnode)
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	302 pkk_printi(indent, f"{lnode.tag} \"{stmp}\"\n")
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	303 else:
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	304 if isinstance(lnode.text, str):
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	305 textstr = pkk_str_clean(lnode.text).strip()
d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	306 if textstr != "":
d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	307 textstr = " \""+ textstr +"\""
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	308 else:
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	309 textstr = ""
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	310
34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	311 if len(lnode.attrib) > 0:
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	312 attrstr = " "+ str(lnode.attrib)
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	313 else:
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	314 attrstr = ""
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	315
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	316 pkk_printi(indent, f"{lnode.tag}{attrstr}{textstr}\n")
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	317 for qnode in lnode.findall("./*"):
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	318 pkk_dump_recursive(indent + 1, qnode)
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	319
34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	320
17 6fa24c711f86 Add some comments. Matti Hamalainen <ccr@tnsp.org> parents: 16 diff changeset	321 ## Output item(s) under given node with given format string
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	322 def pkk_get_subs(indent, dnode, dsub, dfmtname):
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	323 dfmt = pkk_get_fmt(dfmtname)
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	324 ostr = ""
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	325 for qnode in dnode.findall(dsub):
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	326 ostr += dfmt.format(
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	327 text=pkk_node_to_text(qnode),
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	328 indent=pkk_geti(indent))
6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	329 return ostr
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	330
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	331
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	332 def pkk_get_list_str(dlist, dprefix, dfilter):
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	333 if len(dlist) > 0:
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	334 if dfilter:
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	335 tfmt = pkk_get_fmt(dprefix + "_list_item")
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	336 tlist = [tfmt.format(text=i) for i in dlist]
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	337 else:
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	338 tlist = dlist
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	339
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	340 return pkk_get_fmt(dprefix + "_list").format(
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	341 alist=pkk_get_fmt(dprefix + "_list_sep").join(tlist))
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	342 else:
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	343 return pkk_get_fmt(dprefix + "_list_empty")
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	344
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	345
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	346 ## Output a main "Headword" or "Sense" node
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	347 def pkk_get_sense(indent, dnode):
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	348 # Definition for this sense
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	349 ostr = pkk_get_subs(indent, dnode, "./Definition", "definition_item")
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	350
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	351 # Examples for this sense
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	352 for wnode in dnode.findall("./ExampleBlock/ExampleCtn"):
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	353 geolist = []
24 8c8e8e4504bb Remove verbosity option as it is not really used. Matti Hamalainen <ccr@tnsp.org> parents: 23 diff changeset	354 for qnode in wnode.findall("./FreeTopic[@type='levikki']/GeographicalUsage"):
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	355 geolist.append(pkk_get_fmt("example_geo_list_item").format(
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	356 text=pkk_node_to_text(qnode),
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	357 tclass=qnode.attrib["class"]))
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	358
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	359 ostr += pkk_get_fmt("example_item").format(
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	360 text=pkk_node_to_text(wnode.find("./Example")),
54 884770576e74 Rename a token. Matti Hamalainen <ccr@tnsp.org> parents: 53 diff changeset	361 geo_list=pkk_get_list_str(geolist, "example_geo", False),
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	362 indent=pkk_geti(indent + 1))
6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	363
6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	364 return ostr
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	365
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	366
17 6fa24c711f86 Add some comments. Matti Hamalainen <ccr@tnsp.org> parents: 16 diff changeset	367 ## Output one "DictionaryEntry" node
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	368 def pkk_output_node(indent, dnode):
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	369
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	370 for wnode in dnode.findall("./HeadwordCtn"):
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	371 # Get head word
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	372 headword = pkk_node_to_text(wnode.find("./Headword"))
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	373
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	374 # Collect search forms
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	375 srchlist = []
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	376 for qnode in wnode.findall("./SearchForm"):
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	377 srchlist.append(pkk_node_to_text(qnode))
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	378
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	379 # Remove dupe if headword is also in srchlist
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	380 if headword in srchlist:
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	381 srchlist.remove(headword)
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	382
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	383 # Remove other duplicates and sort
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	384 srchlist = list(set(srchlist))
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	385 srchlist.sort(reverse=False, key=lambda attr: (attr, len(attr)))
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	386
44 d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	387 # Get hyphenation note, if any
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	388 hnode = wnode.find("./Hyphenation")
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	389 if hnode != None:
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	390 hyphenation = pkk_get_fmt("hyphenation").format(
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	391 text=pkk_node_to_text(hnode),
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	392 indent=pkk_geti(indent + 1))
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	393 else:
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	394 hyphenation = pkk_get_fmt("no_hyphenation").format(
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	395 indent=pkk_geti(indent + 1))
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	396
18 ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	397 # Create list with grammatical attributes (noun, verb, etc.)
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	398 attrlist = []
18 ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	399 for pnode in wnode.findall("./PartOfSpeechCtn/PartOfSpeech"):
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	400 attrlist.append(pnode.attrib["freeValue"])
18 ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	401
ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	402 for pnode in wnode.findall("./GrammaticalNote"):
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	403 attrlist.append(pkk_node_to_text(pnode))
18 ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	404
ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	405 # Remove duplicates and sort the list
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	406 attrlist = list(set(attrlist))
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	407 attrlist.sort(reverse=False, key=lambda attr: (attr, len(attr)))
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	408
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	409 # Get main "sense"
6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	410 msense = pkk_get_sense(indent + 1, wnode)
6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	411
6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	412 # Print any other "senses"
6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	413 index = 1
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	414 senselist = []
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	415 for znode in dnode.findall("./SenseGrp"):
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	416 senselist.append(pkk_get_fmt("sense_list_item").format(
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	417 index=index,
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	418 text=pkk_get_sense(indent + 2, znode),
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	419 indent=pkk_geti(indent + 1)))
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	420 index += 1
6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	421
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	422 # Print the headword and attributes if any
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	423 pkk_print(pkk_get_fmt("word_item").format(
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	424 word=headword,
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	425 attr=pkk_get_list_str(attrlist, "word_attr", True),
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	426 search=pkk_get_list_str(srchlist, "search", True),
44 d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	427 hyphenation=hyphenation,
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	428 main_sense=msense,
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	429 other_senses=pkk_get_list_str(senselist, "sense", False),
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	430 indent=pkk_geti(indent)))
18 ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	431
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	432
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	433 ###
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	434 ### Main program starts
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	435 ###
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	436 signal.signal(signal.SIGINT, pkk_signal_handler)
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	437
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	438 optparser = argparse.ArgumentParser(
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	439 description="lxmldump - Convert and dump ISO/FDIS 1951 XML file data",
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	440 usage="%(prog)s [options] <input xml file(s)>",
31 4cbefe4c6f53 Improve help, list the default Ptr format strings for each mode. Matti Hamalainen <ccr@tnsp.org> parents: 30 diff changeset	441 add_help=False
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	442 )
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	443
22 76856fc4e5fa Clean up argument handling. Matti Hamalainen <ccr@tnsp.org> parents: 21 diff changeset	444 optparser.add_argument("filenames",
76856fc4e5fa Clean up argument handling. Matti Hamalainen <ccr@tnsp.org> parents: 21 diff changeset	445 type=str, action="extend", nargs="*",
76856fc4e5fa Clean up argument handling. Matti Hamalainen <ccr@tnsp.org> parents: 21 diff changeset	446 metavar="filename",
76856fc4e5fa Clean up argument handling. Matti Hamalainen <ccr@tnsp.org> parents: 21 diff changeset	447 help="XML filename(s)")
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	448
31 4cbefe4c6f53 Improve help, list the default Ptr format strings for each mode. Matti Hamalainen <ccr@tnsp.org> parents: 30 diff changeset	449 optparser.add_argument("-h", "--help",
4cbefe4c6f53 Improve help, list the default Ptr format strings for each mode. Matti Hamalainen <ccr@tnsp.org> parents: 30 diff changeset	450 dest="show_help",
4cbefe4c6f53 Improve help, list the default Ptr format strings for each mode. Matti Hamalainen <ccr@tnsp.org> parents: 30 diff changeset	451 action="store_true",
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	452 help="show this help message")
31 4cbefe4c6f53 Improve help, list the default Ptr format strings for each mode. Matti Hamalainen <ccr@tnsp.org> parents: 30 diff changeset	453
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	454 optparser.add_argument("-m", "--mode",
22 76856fc4e5fa Clean up argument handling. Matti Hamalainen <ccr@tnsp.org> parents: 21 diff changeset	455 dest="mode",
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	456 action=pkk_set_mode,
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	457 default=PKK_MODE_NORMAL,
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	458 help="set output mode (see below)")
35 5aafa87dbec2 Some Anki-related stuff, not finalized and thus hidden. Matti Hamalainen <ccr@tnsp.org> parents: 34 diff changeset	459
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	460 optparser.add_argument("-s", "--set",
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	461 action=pkk_set_value,
38 0e586b4ab62c Cosmetic adjustments to help. Matti Hamalainen <ccr@tnsp.org> parents: 37 diff changeset	462 metavar="ID=STR",
0e586b4ab62c Cosmetic adjustments to help. Matti Hamalainen <ccr@tnsp.org> parents: 37 diff changeset	463 help='set format string (see below)')
32 cfecd039506e Make word attribute separator string configurable. Matti Hamalainen <ccr@tnsp.org> parents: 31 diff changeset	464
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	465 optparser.add_argument("-n", "--normalize",
22 76856fc4e5fa Clean up argument handling. Matti Hamalainen <ccr@tnsp.org> parents: 21 diff changeset	466 dest="normalize",
29 f91ef7d7615b Use store_true feature of argparse. Matti Hamalainen <ccr@tnsp.org> parents: 28 diff changeset	467 action="store_true",
22 76856fc4e5fa Clean up argument handling. Matti Hamalainen <ccr@tnsp.org> parents: 21 diff changeset	468 help="output NFC normalized Unicode")
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	469
f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	470 optparser.add_argument("-a", "--annotate",
22 76856fc4e5fa Clean up argument handling. Matti Hamalainen <ccr@tnsp.org> parents: 21 diff changeset	471 dest="annotate",
29 f91ef7d7615b Use store_true feature of argparse. Matti Hamalainen <ccr@tnsp.org> parents: 28 diff changeset	472 action="store_true",
22 76856fc4e5fa Clean up argument handling. Matti Hamalainen <ccr@tnsp.org> parents: 21 diff changeset	473 help="annotate strings")
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	474
22 76856fc4e5fa Clean up argument handling. Matti Hamalainen <ccr@tnsp.org> parents: 21 diff changeset	475 optparser.add_argument("-i", "--indent",
76856fc4e5fa Clean up argument handling. Matti Hamalainen <ccr@tnsp.org> parents: 21 diff changeset	476 dest="indent",
76856fc4e5fa Clean up argument handling. Matti Hamalainen <ccr@tnsp.org> parents: 21 diff changeset	477 type=int, choices=range(0, 32), default=4,
38 0e586b4ab62c Cosmetic adjustments to help. Matti Hamalainen <ccr@tnsp.org> parents: 37 diff changeset	478 metavar="N",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	479 help='set indentation level (default: %(default)s)')
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	480
35 5aafa87dbec2 Some Anki-related stuff, not finalized and thus hidden. Matti Hamalainen <ccr@tnsp.org> parents: 34 diff changeset	481 optparser.add_argument("-p", "--debug",
5aafa87dbec2 Some Anki-related stuff, not finalized and thus hidden. Matti Hamalainen <ccr@tnsp.org> parents: 34 diff changeset	482 dest="debug",
5aafa87dbec2 Some Anki-related stuff, not finalized and thus hidden. Matti Hamalainen <ccr@tnsp.org> parents: 34 diff changeset	483 action="store_true",
5aafa87dbec2 Some Anki-related stuff, not finalized and thus hidden. Matti Hamalainen <ccr@tnsp.org> parents: 34 diff changeset	484 help=argparse.SUPPRESS)
5aafa87dbec2 Some Anki-related stuff, not finalized and thus hidden. Matti Hamalainen <ccr@tnsp.org> parents: 34 diff changeset	485
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	486
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	487 ### Parse arguments
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	488 pkk_cfg = optparser.parse_args()
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	489
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	490
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	491 ### Show help if needed
31 4cbefe4c6f53 Improve help, list the default Ptr format strings for each mode. Matti Hamalainen <ccr@tnsp.org> parents: 30 diff changeset	492 if len(pkk_cfg.filenames) == 0 or pkk_cfg.show_help:
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	493 optparser.print_help()
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	494 print(u"\nAvailable output modes:")
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	495 print(" " + ", ".join(pkk_modes_list.values()))
31 4cbefe4c6f53 Improve help, list the default Ptr format strings for each mode. Matti Hamalainen <ccr@tnsp.org> parents: 30 diff changeset	496
41 98c85c0b5159 Improve help. Matti Hamalainen <ccr@tnsp.org> parents: 40 diff changeset	497 if pkk_cfg.mode not in [PKK_MODE_NORMAL, PKK_MODE_ANKI]:
98c85c0b5159 Improve help. Matti Hamalainen <ccr@tnsp.org> parents: 40 diff changeset	498 pkk_cfg.mode = PKK_MODE_NORMAL
98c85c0b5159 Improve help. Matti Hamalainen <ccr@tnsp.org> parents: 40 diff changeset	499
98c85c0b5159 Improve help. Matti Hamalainen <ccr@tnsp.org> parents: 40 diff changeset	500 print(u"\nAvailable format strings and values (mode '{}'):".format(
98c85c0b5159 Improve help. Matti Hamalainen <ccr@tnsp.org> parents: 40 diff changeset	501 pkk_modes_list[pkk_cfg.mode]))
98c85c0b5159 Improve help. Matti Hamalainen <ccr@tnsp.org> parents: 40 diff changeset	502
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	503 for mid in pkk_mode_defaults:
41 98c85c0b5159 Improve help. Matti Hamalainen <ccr@tnsp.org> parents: 40 diff changeset	504 stmp = pkk_get_value(mid).replace("\\", "\\\\").replace("\n", "\\n")
42 508de0f6836b Adjust help output. Matti Hamalainen <ccr@tnsp.org> parents: 41 diff changeset	505 print(u" {:22s} : '{}'".format(mid, stmp))
41 98c85c0b5159 Improve help. Matti Hamalainen <ccr@tnsp.org> parents: 40 diff changeset	506
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	507 sys.exit(0)
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	508
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	509
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	510 ### Handle each input file
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	511 for filename in pkk_cfg.filenames:
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	512 # Parse XML file into element tree
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	513 try:
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	514 uxml = xmlET.parse(filename)
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	515 except Exception as e:
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	516 pkk_fatal(u"SVG/XML parsing failed: {0}".format(str(e)))
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	517
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	518 # Dump output
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	519 try:
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	520 xroot = uxml.getroot()
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	521 for dnode in xroot.findall("./DictionaryEntry"):
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	522
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	523 if pkk_cfg.debug and dnode.attrib["identifier"] not in pkk_debug_list:
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	524 continue
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	525
35 5aafa87dbec2 Some Anki-related stuff, not finalized and thus hidden. Matti Hamalainen <ccr@tnsp.org> parents: 34 diff changeset	526 if pkk_cfg.mode in [PKK_MODE_NORMAL, PKK_MODE_ANKI]:
19 7c6eb57798bd Ja niin. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	527 try:
7c6eb57798bd Ja niin. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	528 pkk_output_node(0, dnode)
7c6eb57798bd Ja niin. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	529 except Exception as e:
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	530 print("")
19 7c6eb57798bd Ja niin. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	531 pkk_dump_recursive(0, dnode)
7c6eb57798bd Ja niin. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	532 print(str(e))
7c6eb57798bd Ja niin. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	533 sys.exit(0)
21 7ef08e05a5bf Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	534 elif pkk_cfg.mode == PKK_MODE_DUMP:
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	535 pkk_dump_recursive(0, dnode)
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	536 print("")
21 7ef08e05a5bf Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	537 elif pkk_cfg.mode == PKK_MODE_XML:
47 2e8282f1a837 Fix XML node output. Matti Hamalainen <ccr@tnsp.org> parents: 46 diff changeset	538 pkk_print(xmlET.tostring(dnode, encoding="utf8").decode("utf8") + "\n\n")
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	539 else:
10 013f0cd9e5b3 More work. Matti Hamalainen <ccr@tnsp.org> parents: 9 diff changeset	540 pkk_fatal("Invalid operation mode?")
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	541
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	542 except (BrokenPipeError, IOError) as e:
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	543 sys.stderr.close()
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	544 sys.exit(1)
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	545
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	546 sys.exit(0)

Mercurial > hg > lxmldump

annotate lxmldump.py @ 56:71e66eca1e23