lxmldump: lxmldump.py annotate

annotate lxmldump.py @ 84:1cb41a6cd1cd default tip

Cleanups.

author	Matti Hamalainen <ccr@tnsp.org>
date	Tue, 31 Oct 2023 20:19:14 +0200
parents	a42e0ca2277f
children

rev	line source
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	1 #!/usr/bin/python3 -B
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	2 # coding=utf-8
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	3 ###
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	4 ### lxmldump - Convert and dump ISO/FDIS 1951 XML file data
4 60b789dfee32 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 1 diff changeset	5 ### Programmed and designed by Matti 'ccr' Hämäläinen <ccr@tnsp.org>
81 4fb259e345c2 Bump copyright years. Matti Hamalainen <ccr@tnsp.org> parents: 80 diff changeset	6 ### (C) Copyright 2021-2022 Tecnic Software productions (TNSP)
4 60b789dfee32 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 1 diff changeset	7 ###
23 3ef1c5463b8f Add license. Matti Hamalainen <ccr@tnsp.org> parents: 22 diff changeset	8 ### Released / distributed under 3-clause BSD license
3ef1c5463b8f Add license. Matti Hamalainen <ccr@tnsp.org> parents: 22 diff changeset	9 ### (see file "COPYING" for more information)
3ef1c5463b8f Add license. Matti Hamalainen <ccr@tnsp.org> parents: 22 diff changeset	10 ###
4 60b789dfee32 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 1 diff changeset	11 ### Python 3.7+ required!
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	12 ###
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	13 import sys
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	14 import signal
67 9833df4573a7 Improve handling of value range commandline options. Matti Hamalainen <ccr@tnsp.org> parents: 66 diff changeset	15 import functools
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	16 import re
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	17 import xml.etree.ElementTree as xmlET
5 274b2091137c Some more work on cleaning this up. Matti Hamalainen <ccr@tnsp.org> parents: 4 diff changeset	18 import unicodedata
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	19 import argparse
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	20
82 c876f8872552 Add informative message to the assert requiring Python 3.7+ Matti Hamalainen <ccr@tnsp.org> parents: 81 diff changeset	21 assert sys.version_info >= (3, 7), 'Python >= 3.7 required'
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	22
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	23
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	24 ###
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	25 ### Default settings
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	26 ###
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	27 # Operation modes
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	28 PKK_MODE_NORMAL = 0
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	29 PKK_MODE_DUMP = 1
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	30 PKK_MODE_XML = 2
35 5aafa87dbec2 Some Anki-related stuff, not finalized and thus hidden. Matti Hamalainen <ccr@tnsp.org> parents: 34 diff changeset	31 PKK_MODE_ANKI = 3
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	32
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	33
30 34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	34 pkk_modes_list = {
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	35 PKK_MODE_NORMAL: "normal",
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	36 PKK_MODE_DUMP: "dump",
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	37 PKK_MODE_XML: "xml",
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	38 PKK_MODE_ANKI: "anki",
30 34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	39 }
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	40
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	41
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	42 pkk_mode_defaults = {
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	43 # Default Ptr URL format strings
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	44 "ptr_fmt": {
52 95671cdda422 Remove u specifier. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	45 PKK_MODE_NORMAL: "<PTR:{href}>{text}</PTR>",
95671cdda422 Remove u specifier. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	46 PKK_MODE_ANKI: "<a href='https://kaino.kotus.fi/cgi-bin/kks/karjala.cgi?a={href}'>{text}</a>",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	47 },
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	48
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	49 "word_item": {
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	50 PKK_MODE_NORMAL: "{word}{search}{attr}{hyphenation}:{main_sense}{other_senses}\n",
57 4410e2d1bff3 Adjust Anki output. Matti Hamalainen <ccr@tnsp.org> parents: 56 diff changeset	51 PKK_MODE_ANKI: "\"{word}\"{search}{attr}{hyphenation};{main_sense}{other_senses}\n",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	52 },
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	53 "word_attr_list": {
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	54 PKK_MODE_NORMAL: " ({alist})",
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	55 PKK_MODE_ANKI: " ({alist})",
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	56 },
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	57 "word_attr_list_empty": {
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	58 PKK_MODE_NORMAL: "",
49 d3d4b547f86c Some prodding at Anki output. Matti Hamalainen <ccr@tnsp.org> parents: 48 diff changeset	59 PKK_MODE_ANKI: "",
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	60 },
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	61 "word_attr_list_item": {
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	62 PKK_MODE_NORMAL: "{text}",
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	63 },
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	64 "word_attr_list_sep": {
64 66d585e862b6 Fixes. Matti Hamalainen <ccr@tnsp.org> parents: 63 diff changeset	65 PKK_MODE_NORMAL: "; ",
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	66 PKK_MODE_ANKI: " : ",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	67 },
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	68
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	69 "hyphenation": {
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	70 PKK_MODE_NORMAL: " [hyph: {text}]",
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	71 PKK_MODE_ANKI: " [hyph: {text}]",
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	72 },
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	73 "no_hyphenation": {
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	74 PKK_MODE_NORMAL: "",
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	75 },
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	76
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	77
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	78 "search_list": {
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	79 PKK_MODE_NORMAL: ", {alist}",
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	80 PKK_MODE_ANKI: ", {alist}",
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	81 },
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	82 "search_list_empty": {
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	83 PKK_MODE_NORMAL: "",
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	84 },
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	85 "search_list_item": {
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	86 PKK_MODE_NORMAL: "{text}",
afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	87 PKK_MODE_ANKI: "\"{text}\"",
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	88 },
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	89 "search_list_sep": {
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	90 PKK_MODE_NORMAL: ", ",
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	91 },
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	92
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	93 "main_sense_item": {
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	94 PKK_MODE_NORMAL: "{definition}{example_list}",
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	95 PKK_MODE_ANKI: "{definition}{example_list}",
44 d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	96 },
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	97 "sense_list": {
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	98 PKK_MODE_NORMAL: "{alist}",
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	99 PKK_MODE_ANKI: " \| {alist}",
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	100 },
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	101 "sense_list_empty": {
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	102 PKK_MODE_NORMAL: "",
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	103 },
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	104 "sense_list_item": {
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	105 PKK_MODE_NORMAL: "#{index}:{definition}{example_list}",
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	106 PKK_MODE_ANKI: "#{index}:{definition}{example_list}",
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	107 },
301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	108 "sense_list_sep": {
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	109 PKK_MODE_NORMAL: " \| ",
62 1932f588743f Fixes. Matti Hamalainen <ccr@tnsp.org> parents: 61 diff changeset	110 PKK_MODE_ANKI: " \| ",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	111 },
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	112
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	113 "definition_item": {
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	114 PKK_MODE_NORMAL: " {text}",
57 4410e2d1bff3 Adjust Anki output. Matti Hamalainen <ccr@tnsp.org> parents: 56 diff changeset	115 PKK_MODE_ANKI: " ? \"{text}\"",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	116 },
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	117
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	118 "example_list": {
afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	119 PKK_MODE_NORMAL: " \| {alist}",
afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	120 },
afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	121 "example_list_empty": {
afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	122 PKK_MODE_NORMAL: "",
afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	123 },
afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	124 "example_list_item": {
afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	125 PKK_MODE_NORMAL: "\"{text}\"{geo_list}",
57 4410e2d1bff3 Adjust Anki output. Matti Hamalainen <ccr@tnsp.org> parents: 56 diff changeset	126 PKK_MODE_ANKI: " * \"{text}\"{geo_list}",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	127 },
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	128 "example_list_sep": {
afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	129 PKK_MODE_NORMAL: " ; ",
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	130 PKK_MODE_ANKI: "",
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	131 },
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	132
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	133 "example_geo_list": {
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	134 PKK_MODE_NORMAL: " ({alist})",
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	135 },
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	136 "example_geo_list_empty": {
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	137 PKK_MODE_NORMAL: "",
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	138 },
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	139 "example_geo_list_item": {
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	140 PKK_MODE_NORMAL: "{text} [{tclass}]",
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	141 },
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	142 "example_geo_list_sep": {
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	143 PKK_MODE_NORMAL: ", ",
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	144 },
30 34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	145 }
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	146
34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	147
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	148 # Element annotation mappings
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	149 pkk_element_annotation_map = {
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	150 "Fragment" : {
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	151 PKK_MODE_NORMAL: ["<", ">"],
35 5aafa87dbec2 Some Anki-related stuff, not finalized and thus hidden. Matti Hamalainen <ccr@tnsp.org> parents: 34 diff changeset	152 PKK_MODE_ANKI: ["<", ">"],
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	153 },
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	154 }
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	155
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	156
28 3442b8700da7 Comments. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	157 # List of words in kks1/ useful for debugging, option -p
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	158 pkk_debug_list = [
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	159 "ahavakkaine",
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	160 "ahavakala",
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	161 "ahavakoittuo",
64 66d585e862b6 Fixes. Matti Hamalainen <ccr@tnsp.org> parents: 63 diff changeset	162 "ahava",
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	163 "ahvaliha",
9 2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	164 "aloilleh",
18 ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	165 "hanjahtoakseh",
19 7c6eb57798bd Ja niin. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	166 "akkalisto",
33 3dcf8ac43bda Add one word to debug list. Matti Hamalainen <ccr@tnsp.org> parents: 32 diff changeset	167 "alto-",
50 5b22ffdab0ce Adjustment. Matti Hamalainen <ccr@tnsp.org> parents: 49 diff changeset	168 "allot-",
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	169 ]
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	170
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	171
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	172 pkk_settings = {}
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	173
21 7ef08e05a5bf Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	174
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	175 ###
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	176 ### Misc. helper functions, etc
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	177 ###
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	178
13 3bd772fd6a50 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 12 diff changeset	179 ## Print string to stdout using normalized Unicode if enabled
5 274b2091137c Some more work on cleaning this up. Matti Hamalainen <ccr@tnsp.org> parents: 4 diff changeset	180 def pkk_print(smsg):
12 d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	181 try:
20 f274504eafd0 Use Python argparse module instead of custom self-rolled argument parser. Matti Hamalainen <ccr@tnsp.org> parents: 19 diff changeset	182 if pkk_cfg.normalize:
12 d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	183 sys.stdout.write(unicodedata.normalize("NFC", smsg))
d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	184 else:
d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	185 sys.stdout.write(smsg)
d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	186
d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	187 except (BrokenPipeError, IOError) as e:
d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	188 sys.stderr.close()
d50e71642be7 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 11 diff changeset	189
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	190
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	191 ## Get indentation string
bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	192 def pkk_geti(indent):
bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	193 return " " * pkk_cfg.indent * indent
bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	194
bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	195
13 3bd772fd6a50 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 12 diff changeset	196 ## Print string with indentation
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	197 def pkk_printi(indent, smsg):
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	198 pkk_print(pkk_geti(indent) + smsg)
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	199
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	200
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	201 ## Fatal error handler
76 8471722f70e0 Fix previous cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 75 diff changeset	202 def pkk_fatal(msg):
8471722f70e0 Fix previous cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 75 diff changeset	203 print(f"ERROR: {msg}", file=sys.stderr)
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	204 sys.exit(1)
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	205
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	206
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	207 ## Handler for SIGINT signals
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	208 def pkk_signal_handler(signal, frame):
84 1cb41a6cd1cd Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 83 diff changeset	209 print("\nQuitting due to SIGINT / Ctrl+C!", file=sys.stderr)
1cb41a6cd1cd Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 83 diff changeset	210 sys.exit(2)
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	211
bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	212
67 9833df4573a7 Improve handling of value range commandline options. Matti Hamalainen <ccr@tnsp.org> parents: 66 diff changeset	213 def pkk_arg_range(vstr, vmin, vmax):
78 1e1d478f4845 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 77 diff changeset	214 try:
1e1d478f4845 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 77 diff changeset	215 value = int(vstr)
1e1d478f4845 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 77 diff changeset	216 except Exception as e:
1e1d478f4845 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 77 diff changeset	217 raise argparse.ArgumentTypeError(f"value '{vstr}' is not an integer")
1e1d478f4845 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 77 diff changeset	218
67 9833df4573a7 Improve handling of value range commandline options. Matti Hamalainen <ccr@tnsp.org> parents: 66 diff changeset	219 value = int(vstr)
9833df4573a7 Improve handling of value range commandline options. Matti Hamalainen <ccr@tnsp.org> parents: 66 diff changeset	220 if value < vmin or value > vmax:
77 fea973d0bc76 Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 76 diff changeset	221 raise argparse.ArgumentTypeError(f"value {value} not in range {vmin}-{vmax}")
67 9833df4573a7 Improve handling of value range commandline options. Matti Hamalainen <ccr@tnsp.org> parents: 66 diff changeset	222 else:
9833df4573a7 Improve handling of value range commandline options. Matti Hamalainen <ccr@tnsp.org> parents: 66 diff changeset	223 return value
9833df4573a7 Improve handling of value range commandline options. Matti Hamalainen <ccr@tnsp.org> parents: 66 diff changeset	224
9833df4573a7 Improve handling of value range commandline options. Matti Hamalainen <ccr@tnsp.org> parents: 66 diff changeset	225
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	226 class pkk_set_mode(argparse.Action):
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	227 def __call__(self, parser, namespace, value, option_string=None):
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	228 lvalue = value.strip().lower()
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	229 for mode in pkk_modes_list:
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	230 if pkk_modes_list[mode] == lvalue:
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	231 setattr(namespace, self.dest, mode)
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	232 return
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	233
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	234 pkk_fatal(f"Invalid output mode '{lvalue}'.")
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	235
f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	236
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	237 class pkk_set_value(argparse.Action):
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	238
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	239 rexpr = re.compile(r'\s(\w+)\s=\s(.)\s*')
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	240
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	241 def __call__(self, parser, namespace, values, option_string=None):
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	242 rmatch = re.match(self.rexpr, values)
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	243 if rmatch:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	244 rid = rmatch.group(1).lower().replace("-", "_")
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	245 rval = rmatch.group(2)
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	246 if rid in pkk_mode_defaults:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	247 pkk_settings[rid] = rval
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	248 else:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	249 pkk_fatal(f"Invalid option '{option_string} {values}': No such ID '{rid}'.")
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	250 else:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	251 pkk_fatal(f"Invalid option '{option_string} {values}': Expected id=value.")
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	252
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	253
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	254 ## Get mode if it exists
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	255 def pkk_test_value(mid):
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	256 if mid in pkk_mode_defaults:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	257 if pkk_cfg.mode in pkk_mode_defaults[mid]:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	258 mmode = pkk_cfg.mode
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	259 else:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	260 mmode = PKK_MODE_NORMAL
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	261
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	262 if mmode in pkk_mode_defaults[mid]:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	263 return mmode
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	264 else:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	265 return None
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	266 else:
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	267 return None
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	268
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	269
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	270 ## Get default value per mode
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	271 def pkk_get_value(mid):
83 a42e0ca2277f Use is/is not for comparing to None. Matti Hamalainen <ccr@tnsp.org> parents: 82 diff changeset	272 if mid in pkk_settings and pkk_settings[mid] is not None:
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	273 return pkk_settings[mid]
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	274
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	275 mmode = pkk_test_value(mid)
83 a42e0ca2277f Use is/is not for comparing to None. Matti Hamalainen <ccr@tnsp.org> parents: 82 diff changeset	276 if mmode is None:
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	277 pkk_fatal(f"Internal error: No mode for ID '{mid}'.")
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	278
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	279 return pkk_mode_defaults[mid][mmode]
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	280
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	281
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	282 def pkk_get_fmt(mid):
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	283 return pkk_get_value(mid).replace("\\n", "\n")
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	284
4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	285
71 8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	286 class PKKArgumentParser(argparse.ArgumentParser):
8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	287 def print_help(self):
8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	288 print("lxmldump - Convert and dump ISO/FDIS 1951 XML file data\n"
81 4fb259e345c2 Bump copyright years. Matti Hamalainen <ccr@tnsp.org> parents: 80 diff changeset	289 "(C) Copyright 2021-2022 Matti 'ccr' Hämäläinen <ccr@tnsp.org>\n")
4fb259e345c2 Bump copyright years. Matti Hamalainen <ccr@tnsp.org> parents: 80 diff changeset	290
71 8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	291 super().print_help()
8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	292
80 0d9303f30e68 Remove accidentally duplicated class definition. Matti Hamalainen <ccr@tnsp.org> parents: 79 diff changeset	293 print("\nAvailable output modes:")
0d9303f30e68 Remove accidentally duplicated class definition. Matti Hamalainen <ccr@tnsp.org> parents: 79 diff changeset	294 print(" " + ", ".join(pkk_modes_list.values()))
0d9303f30e68 Remove accidentally duplicated class definition. Matti Hamalainen <ccr@tnsp.org> parents: 79 diff changeset	295
75 cac40b08c555 Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 74 diff changeset	296 def error(self, msg):
71 8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	297 self.print_help()
76 8471722f70e0 Fix previous cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 75 diff changeset	298 print(f"\nERROR: {msg}", file=sys.stderr)
71 8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	299 sys.exit(2)
8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	300
8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	301
28 3442b8700da7 Comments. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	302 ## Annotate given string with prefix and suffix based on tag
26 420f13925f20 Rename a variable. Matti Hamalainen <ccr@tnsp.org> parents: 25 diff changeset	303 def pkk_str_annotate(mtag, mstr):
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	304 if pkk_cfg.annotate and mtag in pkk_element_annotation_map:
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	305 if pkk_cfg.mode in pkk_element_annotation_map[mtag]:
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	306 mmode = pkk_cfg.mode
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	307 else:
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	308 mmode = PKK_MODE_NORMAL
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	309
26 420f13925f20 Rename a variable. Matti Hamalainen <ccr@tnsp.org> parents: 25 diff changeset	310 return pkk_element_annotation_map[mtag][mmode][0] + mstr + pkk_element_annotation_map[mtag][mmode][1]
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	311 else:
26 420f13925f20 Rename a variable. Matti Hamalainen <ccr@tnsp.org> parents: 25 diff changeset	312 return mstr
25 8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	313
8a6738f67106 Factor string annotation into separate function. Matti Hamalainen <ccr@tnsp.org> parents: 24 diff changeset	314
13 3bd772fd6a50 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 12 diff changeset	315 ## Clean string by removing tabs and newlines
10 013f0cd9e5b3 More work. Matti Hamalainen <ccr@tnsp.org> parents: 9 diff changeset	316 def pkk_str_clean(mstr):
60 cbed8ee15701 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 59 diff changeset	317 return re.sub(r'[\n\r\t]', '', mstr).strip()
10 013f0cd9e5b3 More work. Matti Hamalainen <ccr@tnsp.org> parents: 9 diff changeset	318
013f0cd9e5b3 More work. Matti Hamalainen <ccr@tnsp.org> parents: 9 diff changeset	319
28 3442b8700da7 Comments. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	320 ## Format a "Ptr" node as text
9 2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	321 def pkk_ptr_to_text(pnode):
39 f53ea742b57f Refactor output mode handling. Matti Hamalainen <ccr@tnsp.org> parents: 38 diff changeset	322 pfmt = pkk_get_fmt("ptr_fmt")
74 f1fd5a53c341 Fix URLs. Matti Hamalainen <ccr@tnsp.org> parents: 73 diff changeset	323
60 cbed8ee15701 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 59 diff changeset	324 ptext = ("".join(pnode.itertext())).strip()
cbed8ee15701 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 59 diff changeset	325 if pkk_cfg.annotate:
cbed8ee15701 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 59 diff changeset	326 ptext = re.sub(r'\s\.\s$', '', ptext)
74 f1fd5a53c341 Fix URLs. Matti Hamalainen <ccr@tnsp.org> parents: 73 diff changeset	327
f1fd5a53c341 Fix URLs. Matti Hamalainen <ccr@tnsp.org> parents: 73 diff changeset	328 phref = pnode.attrib["{http://www.w3.org/TR/xlink}href"].replace(" ", "+")
f1fd5a53c341 Fix URLs. Matti Hamalainen <ccr@tnsp.org> parents: 73 diff changeset	329
30 34755af2ea1f Make Ptr field URL formatting configurable. Matti Hamalainen <ccr@tnsp.org> parents: 29 diff changeset	330 return pfmt.format(
60 cbed8ee15701 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 59 diff changeset	331 text=ptext,
74 f1fd5a53c341 Fix URLs. Matti Hamalainen <ccr@tnsp.org> parents: 73 diff changeset	332 href=phref)
9 2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	333
2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	334
13 3bd772fd6a50 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 12 diff changeset	335 ## Get text inside a given node
16 285b0820d2c6 Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 15 diff changeset	336 def pkk_node_to_text(lnode):
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	337 stmp = ""
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	338 for pnode in lnode.iter():
9 2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	339 if pnode.tag == "Ptr":
2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	340 stmp += pkk_ptr_to_text(pnode)
2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	341 else:
2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	342 if isinstance(pnode.text, str):
60 cbed8ee15701 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 59 diff changeset	343 stmp += pkk_str_annotate(pnode.tag, pnode.text)
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	344
9 2a8c65d22f86 Moar. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	345 if isinstance(pnode.tail, str):
60 cbed8ee15701 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 59 diff changeset	346 stmp += pnode.tail
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	347
60 cbed8ee15701 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 59 diff changeset	348 return pkk_str_clean(re.sub(r'\s\.\s$', '', stmp))
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	349
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	350
13 3bd772fd6a50 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 12 diff changeset	351 ## Simple recursive dump starting at given node
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	352 def pkk_dump_recursive(indent, lnode):
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	353 if lnode.tag in ["Example"]:
16 285b0820d2c6 Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 15 diff changeset	354 stmp = pkk_node_to_text(lnode)
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	355 pkk_printi(indent, f"{lnode.tag} \"{stmp}\"\n")
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	356 else:
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	357 if isinstance(lnode.text, str):
60 cbed8ee15701 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 59 diff changeset	358 textstr = pkk_str_clean(lnode.text)
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	359 if textstr != "":
d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	360 textstr = " \""+ textstr +"\""
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	361 else:
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	362 textstr = ""
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	363
34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	364 if len(lnode.attrib) > 0:
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	365 attrstr = " "+ str(lnode.attrib)
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	366 else:
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	367 attrstr = ""
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	368
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	369 pkk_printi(indent, f"{lnode.tag}{attrstr}{textstr}\n")
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	370 for qnode in lnode.findall("./*"):
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	371 pkk_dump_recursive(indent + 1, qnode)
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	372
34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	373
17 6fa24c711f86 Add some comments. Matti Hamalainen <ccr@tnsp.org> parents: 16 diff changeset	374 ## Output item(s) under given node with given format string
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	375 def pkk_get_subs(indent, dnode, dsub, dfmtname):
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	376 dfmt = pkk_get_fmt(dfmtname)
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	377 ostr = ""
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	378 for qnode in dnode.findall(dsub):
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	379 ostr += dfmt.format(
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	380 text=pkk_node_to_text(qnode),
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	381 indent=pkk_geti(indent))
6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	382 return ostr
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	383
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	384
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	385 def pkk_get_list_str(dindent, dlist, dprefix, dfilter):
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	386 if len(dlist) > 0:
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	387 if dfilter:
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	388 tfmt = pkk_get_fmt(dprefix + "_list_item")
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	389 tlist = [tfmt.format(text=i) for i in dlist]
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	390 else:
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	391 tlist = dlist
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	392
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	393 return pkk_get_fmt(dprefix + "_list").format(
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	394 alist=pkk_get_fmt(dprefix + "_list_sep").join(tlist),
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	395 indent=pkk_geti(dindent))
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	396 else:
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	397 return pkk_get_fmt(dprefix + "_list_empty").format(
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	398 indent=pkk_geti(dindent))
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	399
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	400
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	401 ## Get definition nand examples from node
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	402 def pkk_get_sense(indent, dnode, dname, dindex):
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	403 exlist = []
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	404 index = 1
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	405 for wnode in dnode.findall("./ExampleBlock/ExampleCtn"):
36 4c8aafff8c5f Refactor output handling to be (mostly) configurable. Matti Hamalainen <ccr@tnsp.org> parents: 35 diff changeset	406 geolist = []
24 8c8e8e4504bb Remove verbosity option as it is not really used. Matti Hamalainen <ccr@tnsp.org> parents: 23 diff changeset	407 for qnode in wnode.findall("./FreeTopic[@type='levikki']/GeographicalUsage"):
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	408 geolist.append(pkk_get_fmt("example_geo_list_item").format(
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	409 text=pkk_node_to_text(qnode),
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	410 tclass=qnode.attrib["class"],
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	411 indent=pkk_geti(indent + 2)))
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	412
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	413 exlist.append(pkk_get_fmt("example_list_item").format(
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	414 text=pkk_node_to_text(wnode.find("./Example")),
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	415 geo_list=pkk_get_list_str(indent + 1, geolist, "example_geo", False),
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	416 indent=pkk_geti(indent + 1),
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	417 index=index))
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	418 index += 1
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	419
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	420 return pkk_get_fmt(dname).format(
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	421 definition=pkk_get_subs(indent, dnode, "./Definition", "definition_item"),
63 afe5cd7626dc Some more work, this may break things a bit. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	422 example_list=pkk_get_list_str(indent, exlist, "example", False),
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	423 indent=pkk_geti(indent),
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	424 index=dindex)
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	425
4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	426
17 6fa24c711f86 Add some comments. Matti Hamalainen <ccr@tnsp.org> parents: 16 diff changeset	427 ## Output one "DictionaryEntry" node
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	428 def pkk_output_node(indent, dnode):
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	429
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	430 for wnode in dnode.findall("./HeadwordCtn"):
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	431 # Get head word
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	432 headword = pkk_node_to_text(wnode.find("./Headword"))
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	433
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	434 # Collect search forms
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	435 srchlist = []
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	436 for qnode in wnode.findall("./SearchForm"):
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	437 srchlist.append(pkk_node_to_text(qnode))
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	438
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	439 # Remove dupe if headword is also in srchlist
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	440 if headword in srchlist:
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	441 srchlist.remove(headword)
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	442
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	443 # Remove other duplicates and sort
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	444 srchlist = list(set(srchlist))
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	445 srchlist.sort(reverse=False, key=lambda attr: (attr, len(attr)))
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	446
44 d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	447 # Get hyphenation note, if any
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	448 hnode = wnode.find("./Hyphenation")
83 a42e0ca2277f Use is/is not for comparing to None. Matti Hamalainen <ccr@tnsp.org> parents: 82 diff changeset	449 if hnode is not None:
44 d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	450 hyphenation = pkk_get_fmt("hyphenation").format(
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	451 text=pkk_node_to_text(hnode),
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	452 indent=pkk_geti(indent + 1))
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	453 else:
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	454 hyphenation = pkk_get_fmt("no_hyphenation").format(
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	455 indent=pkk_geti(indent + 1))
d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	456
18 ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	457 # Create list with grammatical attributes (noun, verb, etc.)
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	458 attrlist = []
18 ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	459 for pnode in wnode.findall("./PartOfSpeechCtn/PartOfSpeech"):
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	460 attrlist.append(pnode.attrib["freeValue"])
18 ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	461
ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	462 for pnode in wnode.findall("./GrammaticalNote"):
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	463 attrlist.append(pkk_node_to_text(pnode))
18 ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	464
ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	465 # Remove duplicates and sort the list
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	466 attrlist = list(set(attrlist))
8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	467 attrlist.sort(reverse=False, key=lambda attr: (attr, len(attr)))
27 d77ab8a300b1 Use Python 3.6/3.7 format strings where we can. Matti Hamalainen <ccr@tnsp.org> parents: 26 diff changeset	468
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	469 # Get main "sense"
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	470 msense = pkk_get_sense(indent + 1, wnode, "main_sense_item", 0)
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	471
6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	472 # Print any other "senses"
6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	473 index = 1
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	474 senselist = []
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	475 for znode in dnode.findall("./SenseGrp"):
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	476 senselist.append(pkk_get_sense(indent + 1, znode, "sense_list_item", index))
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	477 index += 1
6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	478
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	479 # Print the headword and attributes if any
55 301452a71cc7 Make sense output formatting more list-like, rename some items. Matti Hamalainen <ccr@tnsp.org> parents: 51 diff changeset	480 pkk_print(pkk_get_fmt("word_item").format(
43 8ed576574712 More improvements to output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 42 diff changeset	481 word=headword,
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	482 attr=pkk_get_list_str(indent + 1, attrlist, "word_attr", True),
9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	483 search=pkk_get_list_str(indent + 1, srchlist, "search", True),
44 d7b4b2fb0214 Add support for hyphenation data. Matti Hamalainen <ccr@tnsp.org> parents: 43 diff changeset	484 hyphenation=hyphenation,
48 6932c3f2bdeb More flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 47 diff changeset	485 main_sense=msense,
61 9c36574199f5 Enhancements to the output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 60 diff changeset	486 other_senses=pkk_get_list_str(indent + 1, senselist, "sense", False),
40 bc8d8ef4a248 More work on output flexibility. Matti Hamalainen <ccr@tnsp.org> parents: 39 diff changeset	487 indent=pkk_geti(indent)))
18 ff959de0f6c8 Add grammatical attributes. Matti Hamalainen <ccr@tnsp.org> parents: 17 diff changeset	488
6 34a89d61dbe7 Merge and cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 5 3 diff changeset	489
78 1e1d478f4845 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 77 diff changeset	490 ###
1e1d478f4845 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 77 diff changeset	491 ### Main program starts
1e1d478f4845 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 77 diff changeset	492 ###
1e1d478f4845 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 77 diff changeset	493 if __name__ == "__main__":
1e1d478f4845 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 77 diff changeset	494 signal.signal(signal.SIGINT, pkk_signal_handler)
1e1d478f4845 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 77 diff changeset	495
1e1d478f4845 Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 77 diff changeset	496 ### Parse arguments
71 8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	497 optparser = PKKArgumentParser(
8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	498 usage="%(prog)s [options] <input xml file(s)>"
70 f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	499 )
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	500
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	501 optparser.add_argument("filenames",
72 0849d65fabce More work on improving help. Matti Hamalainen <ccr@tnsp.org> parents: 71 diff changeset	502 type=str, action="extend", nargs="*",
70 f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	503 metavar="filename",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	504 help="XML filename(s)")
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	505
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	506 optparser.add_argument("-m", "--mode",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	507 dest="mode",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	508 action=pkk_set_mode,
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	509 default=PKK_MODE_NORMAL,
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	510 help="set output mode (see below)")
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	511
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	512 optparser.add_argument("-s", "--set",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	513 action=pkk_set_value,
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	514 metavar="ID=STR",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	515 help='set format string (see below)')
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	516
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	517 optparser.add_argument("-n", "--normalize",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	518 dest="normalize",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	519 action="store_true",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	520 help="output NFC normalized Unicode")
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	521
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	522 optparser.add_argument("-a", "--annotate",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	523 dest="annotate",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	524 action="store_true",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	525 help="annotate strings")
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	526
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	527 optparser.add_argument("-i", "--indent",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	528 dest="indent",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	529 type=functools.partial(pkk_arg_range, vmin=0, vmax=32), default=4,
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	530 metavar="N",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	531 help='set indentation width (default: %(default)s)')
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	532
71 8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	533 optparser.add_argument("-l", "--list",
8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	534 dest="list_values",
8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	535 action="store_true",
8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	536 help="show list of format strings for current mode")
8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	537
70 f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	538 optparser.add_argument("-p", "--debug",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	539 dest="debug",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	540 action="store_true",
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	541 help=argparse.SUPPRESS)
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	542
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	543 pkk_cfg = optparser.parse_args()
35 5aafa87dbec2 Some Anki-related stuff, not finalized and thus hidden. Matti Hamalainen <ccr@tnsp.org> parents: 34 diff changeset	544
71 8481f8f52a84 More cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 70 diff changeset	545 if pkk_cfg.list_values:
73 4e27677ca5a9 More work on help. Matti Hamalainen <ccr@tnsp.org> parents: 72 diff changeset	546 if pkk_cfg.mode in [PKK_MODE_NORMAL, PKK_MODE_ANKI]:
79 5c7913c838cb Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 78 diff changeset	547 print("Available format strings and values (mode '{}'):".format(
73 4e27677ca5a9 More work on help. Matti Hamalainen <ccr@tnsp.org> parents: 72 diff changeset	548 pkk_modes_list[pkk_cfg.mode]))
70 f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	549
73 4e27677ca5a9 More work on help. Matti Hamalainen <ccr@tnsp.org> parents: 72 diff changeset	550 for mid in pkk_mode_defaults:
4e27677ca5a9 More work on help. Matti Hamalainen <ccr@tnsp.org> parents: 72 diff changeset	551 stmp = pkk_get_value(mid).replace("\\", "\\\\").replace("\n", "\\n")
79 5c7913c838cb Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 78 diff changeset	552 print(" {mid:22s} : '{stmp}'")
73 4e27677ca5a9 More work on help. Matti Hamalainen <ccr@tnsp.org> parents: 72 diff changeset	553 else:
79 5c7913c838cb Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 78 diff changeset	554 print("Mode '{}' does not use format strings.".format(
73 4e27677ca5a9 More work on help. Matti Hamalainen <ccr@tnsp.org> parents: 72 diff changeset	555 pkk_modes_list[pkk_cfg.mode]))
70 f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	556
72 0849d65fabce More work on improving help. Matti Hamalainen <ccr@tnsp.org> parents: 71 diff changeset	557 sys.exit(0)
0849d65fabce More work on improving help. Matti Hamalainen <ccr@tnsp.org> parents: 71 diff changeset	558
0849d65fabce More work on improving help. Matti Hamalainen <ccr@tnsp.org> parents: 71 diff changeset	559 elif len(pkk_cfg.filenames) == 0:
0849d65fabce More work on improving help. Matti Hamalainen <ccr@tnsp.org> parents: 71 diff changeset	560 optparser.print_help()
0849d65fabce More work on improving help. Matti Hamalainen <ccr@tnsp.org> parents: 71 diff changeset	561
70 f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	562 ### Handle each input file
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	563 for filename in pkk_cfg.filenames:
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	564 # Parse XML file into element tree
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	565 try:
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	566 uxml = xmlET.parse(filename)
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	567 except Exception as e:
79 5c7913c838cb Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 78 diff changeset	568 pkk_fatal(f"SVG/XML parsing failed: {str(e)}")
41 98c85c0b5159 Improve help. Matti Hamalainen <ccr@tnsp.org> parents: 40 diff changeset	569
70 f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	570 # Dump output
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	571 try:
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	572 xroot = uxml.getroot()
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	573 for dnode in xroot.findall("./DictionaryEntry"):
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	574
70 f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	575 if pkk_cfg.debug and dnode.attrib["identifier"] not in pkk_debug_list:
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	576 continue
0 bddf1c283e51 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	577
70 f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	578 if pkk_cfg.mode in [PKK_MODE_NORMAL, PKK_MODE_ANKI]:
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	579 try:
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	580 pkk_output_node(0, dnode)
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	581 except Exception as e:
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	582 print("")
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	583 pkk_dump_recursive(0, dnode)
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	584 print(str(e))
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	585 sys.exit(0)
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	586 elif pkk_cfg.mode == PKK_MODE_DUMP:
19 7c6eb57798bd Ja niin. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	587 pkk_dump_recursive(0, dnode)
70 f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	588 print("")
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	589 elif pkk_cfg.mode == PKK_MODE_XML:
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	590 pkk_print(xmlET.tostring(dnode, encoding="utf8").decode("utf8") + "\n\n")
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	591 else:
79 5c7913c838cb Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 78 diff changeset	592 pkk_fatal(f"Invalid operation mode '{pkk_cfg.mode}'.")
7 4b4299b62f7f Moar work. Matti Hamalainen <ccr@tnsp.org> parents: 6 diff changeset	593
70 f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	594 except (BrokenPipeError, IOError) as e:
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	595 sys.stderr.close()
f887d8a68e5e Cleanups, move argparser initialization into a function. Matti Hamalainen <ccr@tnsp.org> parents: 68 diff changeset	596 sys.exit(1)

Mercurial > hg > lxmldump

annotate lxmldump.py @ 84:1cb41a6cd1cd default tip