egg-tcls: urllog.tcl annotate

annotate urllog.tcl @ 590:2294b73df2cf

urllog: Massive refactoring to use similar message system as other scripts, and general cleanups to the code. This breaks configuration backwards compatibility completely.

author	Matti Hamalainen <ccr@tnsp.org>
date	Wed, 27 Jan 2021 12:01:20 +0200
parents	23f2c71fdb90
children	b11484cb3623

rev	line source
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	1 ##########################################################################
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	2 #
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	3 # URLLog v2.7.0 by Matti 'ccr' Hamalainen <ccr@tnsp.org>
578 14dfb925a64a Bump copyright years. Matti Hamalainen <ccr@tnsp.org> parents: 573 diff changeset	4 # (C) Copyright 2000-2021 Tecnic Software productions (TNSP)
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	5 #
113 077c7383f36f urllog: Add line about the script's license. Matti Hamalainen <ccr@tnsp.org> parents: 112 diff changeset	6 # This script is freely distributable under GNU GPL (version 2) license.
077c7383f36f urllog: Add line about the script's license. Matti Hamalainen <ccr@tnsp.org> parents: 112 diff changeset	7 #
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	8 ##########################################################################
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	9 #
458 cfbe6acc1d73 urllog: tcl-tls 1.7.x (tested with 1.7.13) is now a requirement. It is Matti Hamalainen <ccr@tnsp.org> parents: 457 diff changeset	10 # NOTICE! NOTICE! This script REQUIRES tcl-tls 1.7.13+ if you wish to
cfbe6acc1d73 urllog: tcl-tls 1.7.x (tested with 1.7.13) is now a requirement. It is Matti Hamalainen <ccr@tnsp.org> parents: 457 diff changeset	11 # support SSL/TLS https for URL checking. And you probably do.
cfbe6acc1d73 urllog: tcl-tls 1.7.x (tested with 1.7.13) is now a requirement. It is Matti Hamalainen <ccr@tnsp.org> parents: 457 diff changeset	12 #
50 f69363fc1f61 Update some comments and add a bit of documentation. Matti Hamalainen <ccr@tnsp.org> parents: 49 diff changeset	13 # URL-logger script for EggDrop IRC robot, utilizing SQLite3 database
81 17e542b7985a urllog, quotedb: Improve documentation. Matti Hamalainen <ccr@tnsp.org> parents: 73 diff changeset	14 # This script requires SQLite TCL extension. Under Debian, you need:
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	15 # tcl8.5 (or later) libsqlite3-tcl eggdrop eggdrop-data
50 f69363fc1f61 Update some comments and add a bit of documentation. Matti Hamalainen <ccr@tnsp.org> parents: 49 diff changeset	16 #
479 fc3b6bc37927 urllog: Improve documentation slightly. Matti Hamalainen <ccr@tnsp.org> parents: 473 diff changeset	17 # If you are doing a fresh install, you will need to create the initial
fc3b6bc37927 urllog: Improve documentation slightly. Matti Hamalainen <ccr@tnsp.org> parents: 473 diff changeset	18 # database with the required table schemas. You can do that by running
503 fdd1f0b83685 urllog: Migration from old versions hasn't been supported in reality for a Matti Hamalainen <ccr@tnsp.org> parents: 493 diff changeset	19 # "create_urllog_db.tcl". You also need to set up the configuration in
fdd1f0b83685 urllog: Migration from old versions hasn't been supported in reality for a Matti Hamalainen <ccr@tnsp.org> parents: 493 diff changeset	20 # "config.urllog" file.
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	21 #
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	22 ##########################################################################
13 e06d41fb69d5 Begin work on converting urllog.tcl to use an SQLite3 database instead of flat file. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	23
263 f01d60175c44 urllog: Move configuration to external file. Matti Hamalainen <ccr@tnsp.org> parents: 260 diff changeset	24 ### The configuration should be in config.urllog in same directory
f01d60175c44 urllog: Move configuration to external file. Matti Hamalainen <ccr@tnsp.org> parents: 260 diff changeset	25 ### as this script. Or change the line below to point where ever
f01d60175c44 urllog: Move configuration to external file. Matti Hamalainen <ccr@tnsp.org> parents: 260 diff changeset	26 ### you wish. See "config.urllog.example" for an example config file.
f01d60175c44 urllog: Move configuration to external file. Matti Hamalainen <ccr@tnsp.org> parents: 260 diff changeset	27 source [file dirname [info script]]/config.urllog
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	28
291 54d34d086b47 urllog: Use the utility lib for entity conversion. Matti Hamalainen <ccr@tnsp.org> parents: 289 diff changeset	29 ### Required utillib.tcl
54d34d086b47 urllog: Use the utility lib for entity conversion. Matti Hamalainen <ccr@tnsp.org> parents: 289 diff changeset	30 source [file dirname [info script]]/utillib.tcl
54d34d086b47 urllog: Use the utility lib for entity conversion. Matti Hamalainen <ccr@tnsp.org> parents: 289 diff changeset	31
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	32
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	33 ##########################################################################
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	34 # No need to look below this line
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	35 ##########################################################################
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	36 package require sqlite3
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	37 package require http
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	38
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	39 set urllog_name "URLLog"
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	40 set urllog_version "2.7.0"
578 14dfb925a64a Bump copyright years. Matti Hamalainen <ccr@tnsp.org> parents: 573 diff changeset	41 set urllog_message "$urllog_name v$urllog_version (C) 2000-2021 ccr/TNSP"
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	42
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	43
424 825cac46b1cb Cosmetic / stray trailing whitespace cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 422 diff changeset	44 set urllog_httprep [split "\@\|%40\|{\|%7B\|}\|%7D\|\[\|%5B\|\]\|%5D" "\|"]
561 bdccc83a1c22 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 560 diff changeset	45 set urllog_shorturl_str "ABCDEFGHIJKLNMOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	46
102 5425dc418505 urllog: Entity data is now in UTF-8, but TCL source files are interpreted with current system locale, which may not be UTF-8. We must therefore "convert" the entity mapping string to UTF-8 to be certain of TCL's interpretation of its encoding. Matti Hamalainen <ccr@tnsp.org> parents: 101 diff changeset	47
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	48 #-------------------------------------------------------------------------
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	49 ### Utility functions
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	50 proc urllog_log {umsg} {
267 da239a953e24 urllog: Change some setting names, etc. Matti Hamalainen <ccr@tnsp.org> parents: 264 diff changeset	51 global urllog_log_enable urllog_name
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	52
267 da239a953e24 urllog: Change some setting names, etc. Matti Hamalainen <ccr@tnsp.org> parents: 264 diff changeset	53 if {$urllog_log_enable != 0} {
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	54 putlog "$urllog_name: $umsg"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	55 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	56 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	57
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	58
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	59 proc urllog_qm {uid} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	60 global urllog_messages
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	61
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	62 if {[info exists urllog_messages($uid)]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	63 return $urllog_messages($uid)
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	64 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	65 return $uid
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	66 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	67 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	68
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	69
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	70 proc urllog_smsg {apublic anick achan amsg {aargs {}}} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	71 global urllog_preferredmsg urllog_cmd_name
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	72 set amsg [string map [list "@cmd@" $urllog_cmd_name] $amsg]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	73 utl_msg_args $urllog_preferredmsg $apublic $anick $achan $amsg $aargs
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	74 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	75
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	76
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	77 proc urllog_msg {apublic anick achan aid {aargs {}}} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	78 urllog_smsg $apublic $anick $achan [urllog_qm $aid] $aargs
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	79 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	80
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	81
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	82 proc urllog_verb_msg {apublic anick achan aid {aargs {}}} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	83 global urllog_verbose
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	84 if {$urllog_verbose != 0} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	85 urllog_msg $apublic $anick $achan $aid $aargs
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	86 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	87 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	88
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	89
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	90 proc urllog_isnumber {uarg} {
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	91 foreach i [split $uarg {}] {
65 31c8c4f50aa6 urllog: Improve urllog_isnumber function. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	92 if {![string match \[0-9\] $i]} { return 0 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	93 }
65 31c8c4f50aa6 urllog: Improve urllog_isnumber function. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	94 return 1
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	95 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	96
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	97
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	98 proc urllog_is_enabled {uval} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	99 if {$uval} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	100 return "ON."
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	101 } else {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	102 return "OFF."
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	103 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	104 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	105
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	106
116 4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	107 proc urllog_sanitize_encoding {uencoding} {
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	108 regsub -- "^\[a-z\]\[a-z\]_\[A-Z\]\[A-Z\]\." $uencoding "" uencoding
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	109 set uencoding [string tolower $uencoding]
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	110 regsub -- "^iso-" $uencoding "iso" uencoding
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	111 return $uencoding
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	112 }
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	113
294 2bb9bcfb104a Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 291 diff changeset	114
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	115 proc urllog_get_ss {uindex} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	116 global urllog_shorturl_str
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	117 return [string index $urllog_shorturl_str $uindex]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	118 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	119
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	120
150 52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	121 proc urllog_get_short {utime} {
52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	122 global urllog_shorturl_prefix urllog_shorturl_str
52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	123
52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	124 set ulen [string length $urllog_shorturl_str]
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	125
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	126 set u1 [expr $utime / ($ulen * $ulen)]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	127 set utmp [expr $utime % ($ulen * $ulen)]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	128 set u2 [expr $utmp / $ulen]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	129 set u3 [expr $utmp % $ulen]
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	130
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	131 return "\[ $urllog_shorturl_prefix[urllog_get_ss $u1][urllog_get_ss $u2][urllog_get_ss $u3] \]"
150 52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	132 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	133
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	134
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	135 proc urllog_chop_url {ustr} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	136 global urllog_shorturl_max_orig
68 3762c621d1c3 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 65 diff changeset	137
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	138 if {[string length $ustr] > $urllog_shorturl_max_orig} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	139 return "[string range $ustr 0 $urllog_shorturl_max_orig]..."
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	140 } else {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	141 return $ustr
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	142 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	143 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	144
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	145
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	146 proc urllog_chop_title {ustr} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	147 global urllog_title_max
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	148
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	149 if {[string length $ustr] > $urllog_title_max} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	150 return "[string range $ustr 0 $urllog_title_max]..."
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	151 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	152 return $ustr
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	153 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	154 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	155
241 669842725e2f Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 240 diff changeset	156
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	157 #-------------------------------------------------------------------------
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	158 proc urllog_add_url {urlStr urlNick urlHost urlChan urlTitle} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	159 global urllog_db urllog_shorturl urllog_pub_channels
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	160
18 1e2232135354 More changes for SQLite support. Matti Hamalainen <ccr@tnsp.org> parents: 13 diff changeset	161
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	162 ### Does the URL already exist?
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	163 set usql "SELECT id AS uid, utime AS utime, url AS uurl, user AS uuser, host AS uhost, chan AS uchan, title AS utitle FROM urls WHERE url='[utl_escape $urlStr]'"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	164 urllog_db eval $usql {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	165 urllog_log "URL said by $urlNick ($urlStr) already known"
83 f171a9fb7b7b urllog: Split urllog_add function to urllog_exists for checking whether given URL already exists in the database. Use urllog_exists where appropriate. Matti Hamalainen <ccr@tnsp.org> parents: 82 diff changeset	166
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	167 if {[utl_match_delim_list $urllog_pub_channels $uchan]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	168 if {$urllog_shorturl != 0} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	169 urllog_verb_msg 1 $urlNick $urlChan "url_known_short" [list $uuser $uchan $uhost [utl_ctime $utime] $utitle $uurl [urllog_get_short $uid]]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	170 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	171 urllog_verb_msg 1 $urlNick $urlChan "url_known_long" [list $uuser $uchan $uhost [utl_ctime $utime] $utitle $uurl]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	172 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	173 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	174 return 1
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	175 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	176
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	177 ### Validate title
93 4e02c0219afe urllog: Insert NULL into title column when we didn't get a title. Matti Hamalainen <ccr@tnsp.org> parents: 92 diff changeset	178 if {$urlTitle == ""} {
4e02c0219afe urllog: Insert NULL into title column when we didn't get a title. Matti Hamalainen <ccr@tnsp.org> parents: 92 diff changeset	179 set uins "NULL"
4e02c0219afe urllog: Insert NULL into title column when we didn't get a title. Matti Hamalainen <ccr@tnsp.org> parents: 92 diff changeset	180 } else {
295 141bb4a2b76f utillib: utl_escape (which will be deprecated soon). Matti Hamalainen <ccr@tnsp.org> parents: 294 diff changeset	181 set uins "'[utl_escape $urlTitle]'"
93 4e02c0219afe urllog: Insert NULL into title column when we didn't get a title. Matti Hamalainen <ccr@tnsp.org> parents: 92 diff changeset	182 }
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	183
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	184 ### Attempt to insert into database
295 141bb4a2b76f utillib: utl_escape (which will be deprecated soon). Matti Hamalainen <ccr@tnsp.org> parents: 294 diff changeset	185 set usql "INSERT INTO urls (utime,url,user,host,chan,title) VALUES ([unixtime], '[utl_escape $urlStr]', '[utl_escape $urlNick]', '[utl_escape $urlHost]', '[utl_escape $urlChan]', $uins)"
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	186 if {[catch {urllog_db eval $usql} uerrmsg]} {
83 f171a9fb7b7b urllog: Split urllog_add function to urllog_exists for checking whether given URL already exists in the database. Use urllog_exists where appropriate. Matti Hamalainen <ccr@tnsp.org> parents: 82 diff changeset	187 urllog_log "$uerrmsg on SQL:\n$usql"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	188 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	189 }
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	190 set uid [urllog_db last_insert_rowid]
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	191 urllog_log "Added URL ($urlNick@$urlChan): $urlStr"
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	192
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	193
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	194 ### Let's say something, to confirm that everything went well.
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	195 if {$urllog_shorturl != 0} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	196 set urlShort [urllog_get_short $uid]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	197 set ushort "short"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	198 } else {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	199 set urlShort ""
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	200 set ushort "long"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	201 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	202
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	203 if {[string length $urlTitle] > 0} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	204 set umode "url_added_${ushort}_has_title"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	205 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	206 set umode "url_added_${ushort}_no_title"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	207 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	208
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	209 urllog_verb_msg 1 $urlNick $urlChan $umode [list $urlTitle [urllog_chop_title $urlTitle] $urlStr [urllog_chop_url $urlStr] $urlShort]
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	210 return 1
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	211 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	212
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	213
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	214 #-------------------------------------------------------------------------
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	215 proc urllog_clear_request { urlStatus urlSCode urlCode urlData urlMeta } {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	216
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	217 ### Clear the request data
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	218 upvar $urlStatus ustatus
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	219 upvar $urlSCode uscode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	220 upvar $urlCode ucode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	221 upvar $urlData udata
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	222 upvar $urlMeta umeta
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	223
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	224 unset ustatus
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	225 unset uscode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	226 unset ucode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	227 unset udata
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	228 array unset umeta
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	229 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	230
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	231
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	232 #-------------------------------------------------------------------------
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	233 proc urllog_do_request { urlNick urlChan urlStr urlStatus urlSCode urlCode urlData urlMeta } {
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	234
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	235 upvar $urlStatus ustatus
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	236 upvar $urlSCode uscode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	237 upvar $urlCode ucode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	238 upvar $urlData udata
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	239 upvar $urlMeta umeta
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	240
456 102dc89488af urllog: Improve how http headers are formed. Matti Hamalainen <ccr@tnsp.org> parents: 425 diff changeset	241 set urlHeaders {}
102dc89488af urllog: Improve how http headers are formed. Matti Hamalainen <ccr@tnsp.org> parents: 425 diff changeset	242 lappend urlHeaders "Accept-Encoding" "identity"
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	243 #lappend urlHeaders "Connection" "keep-alive"
456 102dc89488af urllog: Improve how http headers are formed. Matti Hamalainen <ccr@tnsp.org> parents: 425 diff changeset	244
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	245 ### Perform request
456 102dc89488af urllog: Improve how http headers are formed. Matti Hamalainen <ccr@tnsp.org> parents: 425 diff changeset	246 if {[catch {set utoken [::http::geturl $urlStr -timeout 6000 -binary 1 -headers $urlHeaders]} uerrmsg]} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	247 urllog_verb_msg 1 $urlNick $urlChan "err_http_get" [list $urlStr $uerrmsg]
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	248 urllog_log "HTTP request failed: $uerrmsg"
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	249 return 0
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	250 }
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	251
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	252 ### Check status
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	253 set ustatus [::http::status $utoken]
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	254 set uscode [::http::code $utoken]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	255 set ucode [::http::ncode $utoken]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	256
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	257 if {$ustatus != "ok"} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	258 urllog_verb_msg 1 $urlNick $urlChan "err_http_status" [list $urlStr $ustatus $uscode $ucode]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	259 urllog_log "Error in HTTP request: $ustatus / $uscode ($urlStr)"
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	260 return 0
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	261 }
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	262
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	263 ### Get data
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	264 set udata [::http::data $utoken]
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	265 array set umeta [::http::meta $utoken]
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	266 ::http::cleanup $utoken
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	267
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	268 ### Sanitize the metadata KEYS
584 9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	269 foreach {ukey uvalue} [array get umeta] {
9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	270 set ukey [string tolower $ukey]
9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	271 set umeta($ukey) $uvalue
9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	272 }
9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	273
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	274 return 1
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	275 }
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	276
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	277
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	278 #-------------------------------------------------------------------------
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	279 proc urllog_validate_url { urlNick urlChan urlMStr urlMProto urlMHostName } {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	280 global urllog_httprep urllog_shorturl_prefix urllog_shorturl
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	281
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	282 upvar $urlMStr urlStr
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	283 upvar $urlMProto urlProto
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	284 upvar $urlMHostName urlHostName
3 8003090caa35 Lots of code cleanups, add "fixer" for RasiaTube links (which suck) to point directly to Youtube. Matti Hamalainen <ccr@tnsp.org> parents: 0 diff changeset	285
571 d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	286 ### Hack for removing parenthesis around an URL
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	287 if {[regexp {^$(.+)$$} $urlStr urlMatch urlClean]} {
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	288 set urlStr $urlClean
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	289 }
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	290
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	291 if {[regexp {^\[(.+)\]$} $urlStr urlMatch urlClean]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	292 set urlStr $urlClean
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	293 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	294
571 d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	295 ### Clean excess stuff, if any, and attempt to
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	296 ### guess the URL protocol component if it is missing
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	297 if {[regexp "(\[a-z\]+)://\[^ \]+" $urlStr urlMatch urlProto]} {
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	298 set urlStr $urlMatch
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	299 } elseif {[regexp "www\.\[^ \]+" $urlStr urlMatch]} {
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	300 set urlStr "http://$urlMatch"
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	301 } elseif {[regexp "ftp\.\[^ \]+" $urlStr urlMatch]} {
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	302 set urlStr "ftp://$urlMatch"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	303 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	304
95 687bdd74dfac urllog: Check if TLS support is enabled when checking if we can fetch title information via HTTP or SSL/HTTP. Matti Hamalainen <ccr@tnsp.org> parents: 93 diff changeset	305 ### Handle URLs that have an IPv4-address
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	306 if {[regexp "(\[a-z\]+)://(\[0-9\]{1,3})\\.(\[0-9\]{1,3})\\.(\[0-9\]{1,3})\\.(\[0-9\]{1,3})" $urlStr urlMatch urlProto ni1 ni2 ni3 ni4]} {
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	307 # Check if the IP is on local network
572 295c225e3152 urllog: Improve invalid/local IPv4 network check Matti Hamalainen <ccr@tnsp.org> parents: 571 diff changeset	308 if {$ni1 == 127 \|\| $ni1 == 10 \|\| ($ni1 == 192 && $ni2 == 168)} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	309 urllog_verb_msg 1 $urlNick $urlChan "err_url_local_net" [list $urlStr]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	310 urllog_log "URL pointing to local network, ignored (${urlStr})."
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	311 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	312 }
572 295c225e3152 urllog: Improve invalid/local IPv4 network check Matti Hamalainen <ccr@tnsp.org> parents: 571 diff changeset	313 if {$ni1 == 0 \|\| $ni1 >= 255 \|\| $ni2 >= 255 \|\| $ni3 >= 255 \|\| $ni4 >= 255} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	314 urllog_verb_msg 1 $urlNick $urlChan "err_url_invalid_net" [list $urlStr]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	315 urllog_log "URL pointing to invalid network, ignored (${urlStr})."
571 d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	316 return 0
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	317 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	318 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	319
96 e5a6c27be365 urllog: Comments and cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 95 diff changeset	320 ### Check now if we have an ShortURL here ...
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	321 if {[string match "${urllog_shorturl_prefix}*" $urlStr]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	322 urllog_log "Ignoring ShortURL: ${urlStr}"
252 eb2fce89b8ab urllog: Comment out some currently unused code. Matti Hamalainen <ccr@tnsp.org> parents: 251 diff changeset	323 # set uud ""
eb2fce89b8ab urllog: Comment out some currently unused code. Matti Hamalainen <ccr@tnsp.org> parents: 251 diff changeset	324 # set usql "SELECT id AS uid, url AS uurl, user AS uuser, host AS uhost, chan AS uchan, title AS utitle FROM urls WHERE utime=$uud"
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	325 # urllog_db eval $usql {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	326 # urllog_msg 1 $urlNick $urlChan "'$utitle' - $uurl"
252 eb2fce89b8ab urllog: Comment out some currently unused code. Matti Hamalainen <ccr@tnsp.org> parents: 251 diff changeset	327 # return 1
eb2fce89b8ab urllog: Comment out some currently unused code. Matti Hamalainen <ccr@tnsp.org> parents: 251 diff changeset	328 # }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	329 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	330 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	331
95 687bdd74dfac urllog: Check if TLS support is enabled when checking if we can fetch title information via HTTP or SSL/HTTP. Matti Hamalainen <ccr@tnsp.org> parents: 93 diff changeset	332 ### Get URL protocol component
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	333 set urlProto ""
464 506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	334 if {[regexp "(\[a-z\]+)://" $urlStr urlMatch urlProto]} {
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	335 ### Is it a http or ftp url?
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	336 if {$urlProto != "http" && $urlProto != "https" && $urlProto != "ftp"} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	337 urllog_verb_msg 1 $urlNick $urlChan "err_url_proto_class" [list $urlStr $urlProto]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	338 urllog_log "Broken URL: ${urlStr} - unsupported protocol class (${urlProto})."
464 506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	339 return 0
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	340 }
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	341 } else {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	342 urllog_verb_msg 1 $urlNick $urlChan "err_url_proto_no_class" [list $urlStr]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	343 urllog_log "Broken URL: ${urlStr} - no protocol specifier."
464 506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	344 return 0
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	345 }
95 687bdd74dfac urllog: Check if TLS support is enabled when checking if we can fetch title information via HTTP or SSL/HTTP. Matti Hamalainen <ccr@tnsp.org> parents: 93 diff changeset	346
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	347 ### Check the PORT (if the ":" is there)
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	348 set urlRecord [split $urlStr "/"]
a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	349 set urlHostName [lindex $urlRecord 2]
a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	350 set urlPort [lindex [split $urlHostName ":"] end]
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	351
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	352 if {$urlPort != "" && ![urllog_isnumber $urlPort] && $urlPort != $urlHostName} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	353 urllog_verb_msg 1 $urlNick $urlChan "err_url_invalid_port" [list $urlStr $urlPort]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	354 urllog_log "Broken URL: ${urlStr} - illegal or invalid port '${urlPort}'"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	355 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	356 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	357
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	358
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	359 set urlStr [string map $urllog_httprep $urlStr]
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	360 return 1
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	361 }
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	362
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	363
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	364 #-------------------------------------------------------------------------
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	365 proc urllog_handle_redirect {urlNick urlHost urlChan urlRedirLevel urlProto urlHostName urlStr urlStatus urlSCode urlCode urlData urlMeta} {
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	366
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	367 upvar $urlProto uproto
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	368 upvar $urlHostName uhostname
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	369 upvar $urlStr ustr
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	370 upvar $urlStatus ustatus
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	371 upvar $urlSCode uscode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	372 upvar $urlCode ucode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	373 upvar $urlData udata
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	374 upvar $urlMeta umeta
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	375
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	376 ### Was result a redirect?
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	377 if {$ucode >= 301 && $ucode <= 303} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	378 ### Check that we have a location header
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	379 if {![info exists umeta(location)]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	380 urllog_verb_msg 1 $urlNick $urlChan "err_redirect_invalid" [list $ustr $ustatus $uscode $ucode $urlRedirLevel]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	381 urllog_log "Invalid redirect without location header: status=$ustatus, code=$ucode, scode=$uscode, url=$ustr, redirLevel=$urlRedirLevel"
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	382 return 0
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	383 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	384
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	385 ### Fix up location URI
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	386 set nustr $umeta(location)
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	387 if {![regexp "\[a-z\]+://" $nustr]} {
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	388 if {[string range $nustr 0 0] != "/"} {
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	389 append nustr "/"
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	390 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	391 set nustr "${uproto}://${uhostname}${nustr}"
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	392 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	393
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	394 ### Validate the target URI
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	395 urllog_log "Redirection #${urlRedirLevel}: $ustr -> $nustr"
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	396 set ustr $nustr
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	397
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	398 if {![urllog_validate_url $urlNick $urlChan ustr uproto uhostname]} {
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	399 return 0
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	400 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	401
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	402 ### Attempt to fetch redirection target
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	403 urllog_clear_request ustatus uscode ucode udata umeta
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	404 if {![urllog_do_request $urlNick $urlChan $ustr ustatus uscode ucode udata umeta]} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	405 urllog_verb_msg 1 $urlNick $urlChan "err_redirect_fail" [list $ustr $ustatus $uscode $ucode $urlRedirLevel]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	406 urllog_log "Error fetching redirect: status=$ustatus, code=$ucode, scode=$uscode, url=$ustr, redirLevel=$urlRedirLevel"
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	407 return 0
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	408 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	409 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	410
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	411 return 1
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	412 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	413
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	414
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	415 #-------------------------------------------------------------------------
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	416 proc urllog_check_url {urlStr urlNick urlHost urlChan} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	417 global urllog_encoding http_tls_support
581 148c7553c50f urllog: Remove rasiatube hack setting, it's not used. Matti Hamalainen <ccr@tnsp.org> parents: 579 diff changeset	418 global urllog_extra_checks urllog_extra_strict
3 8003090caa35 Lots of code cleanups, add "fixer" for RasiaTube links (which suck) to point directly to Youtube. Matti Hamalainen <ccr@tnsp.org> parents: 0 diff changeset	419
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	420 ### Validate URL compoments, etc.
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	421 set urlProto ""
a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	422 set urlHostName ""
a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	423 if {![urllog_validate_url $urlNick $urlChan urlStr urlProto urlHostName]} {
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	424 return 1
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	425 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	426
267 da239a953e24 urllog: Change some setting names, etc. Matti Hamalainen <ccr@tnsp.org> parents: 264 diff changeset	427 ### Do we perform additional checks?
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	428 if {$urllog_extra_checks == 0 \|\| !(($http_tls_support != 0 && $urlProto == "https") \|\| $urlProto == "http")} {
230 3e3756b113a1 urllog: Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 229 diff changeset	429 # No optional checks, or it's not http/https.
306 9858b93387a2 urllog: 100L. Matti Hamalainen <ccr@tnsp.org> parents: 304 diff changeset	430 if {$urllog_extra_strict == 0} {
304 f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	431 # Strict checking disabled, so add the URL, if it does not exist already.
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	432 urllog_add_url $urlStr $urlNick $urlHost $urlChan ""
304 f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	433 return 1
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	434 } elseif {$http_tls_support == 0 && $urlProto == "https"} {
304 f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	435 # Strict ENABLED: If TLS support is disabled and we have https, do nothing
f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	436 return 1
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	437 } elseif {$urlProto != "http" && $urlProto != "https"} {
304 f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	438 # Strict ENABLED: It's not http, or https
f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	439 return 1
f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	440 }
7 50b52294e93e urllog: Strip &rlm; entities from titles; Some work on SSL/https support. Matti Hamalainen <ccr@tnsp.org> parents: 4 diff changeset	441
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	442 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	443 ### Does the document pointed by the URL exist?
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	444 if {![urllog_do_request $urlNick $urlChan $urlStr ustatus uscode ucode udata umeta]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	445 return 1
116 4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	446 }
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	447
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	448 ### Handle redirects of 3 levels
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	449 if {![urllog_handle_redirect $urlNick $urlHost $urlChan 1 urlProto urlHostName urlStr ustatus uscode ucode udata umeta]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	450 return 1
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	451 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	452
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	453 if {![urllog_handle_redirect $urlNick $urlHost $urlChan 2 urlProto urlHostName urlStr ustatus uscode ucode udata umeta]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	454 return 1
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	455 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	456
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	457 if {![urllog_handle_redirect $urlNick $urlHost $urlChan 3 urlProto urlHostName urlStr ustatus uscode ucode udata umeta]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	458 return 1
116 4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	459 }
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	460
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	461 # Final document
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	462 if {$ucode >= 200 && $ucode <= 205} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	463 set uenc_doc ""
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	464 set uenc_http ""
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	465 set uencoding ""
116 4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	466
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	467 # Get information about specified character encodings
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	468 if {[info exists umeta(content-type)] && [regexp -nocase {charset\s=\s([a-z0-9._-]+)} $umeta(content-type) umatches uenc_http]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	469 # Found character set encoding information in HTTP headers
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	470 }
470 2faf2eb18f26 urllog: Add support for Twitter meta titles. Matti Hamalainen <ccr@tnsp.org> parents: 469 diff changeset	471
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	472 if {[regexp -nocase -- "<meta.\\?content=\"text/html.\\?charset=(\[^\"\])\".\\?/\?>" $udata umatches uenc_doc]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	473 # Found old style HTML meta tag with character set information
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	474 } elseif {[regexp -nocase -- "<meta.\\?charset=\"(\[^\"\])\".\*\?/\?>" $udata umatches uenc_doc]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	475 # Found HTML5 style meta tag with character set information
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	476 }
313 8175ef52889b urllog: Improve URL title functionality. Matti Hamalainen <ccr@tnsp.org> parents: 312 diff changeset	477
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	478 # Make sanitized versions of the encoding strings
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	479 set uenc_http2 [urllog_sanitize_encoding $uenc_http]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	480 set uenc_doc2 [urllog_sanitize_encoding $uenc_doc]
313 8175ef52889b urllog: Improve URL title functionality. Matti Hamalainen <ccr@tnsp.org> parents: 312 diff changeset	481
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	482 # Check if the document has specified encoding
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	483 # KLUDGE!
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	484 set uencoding $uenc_http2
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	485 if {$uencoding == "" && $uenc_doc2 != ""} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	486 set uencoding $uenc_doc2
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	487 } elseif {$uencoding == ""} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	488 # If _NO_ known encoding of any kind, assume the default of iso8859-1
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	489 set uencoding "iso8859-1"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	490 }
150 52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	491
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	492 urllog_log "Charsets: http='$uenc_http', doc='$uenc_doc' / sanitized http='$uenc_http2', doc='$uenc_doc2' -> '$uencoding'"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	493
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	494 # Get the document title, if any
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	495 set urlTitle ""
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	496
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	497 if {[regexp -nocase -- "<meta name=\"twitter:title\" content=\"(.\\?)\"\\s\\/\?>" $udata umatches urlTitle]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	498 # ...
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	499 } elseif {[regexp -nocase -- "<title.\\?>(.\\?)</title>" $udata umatches urlTitle]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	500 # ...
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	501 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	502
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	503 # If facebook, get meta info
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	504 if {[regexp -nocase -- "(http\|https):\/\/www.facebook.com" $urlStr]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	505 if {[regexp -nocase -- "<meta name=\"description\" content=\"(.\*\?)\"" $udata umatches urlTmp]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	506 if {$urlTitle != ""} { append urlTitle " :: " }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	507 append urlTitle $urlTmp
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	508 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	509 }
3 8003090caa35 Lots of code cleanups, add "fixer" for RasiaTube links (which suck) to point directly to Youtube. Matti Hamalainen <ccr@tnsp.org> parents: 0 diff changeset	510
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	511 # If character set conversion is required, do it now
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	512 if {$urlTitle != "" && $uencoding != ""} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	513 if {[catch {set urlTitle [encoding convertfrom $uencoding $urlTitle]} cerrmsg]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	514 urllog_verb_msg 1 $urlNick $urlChan "err_charset" [list $urlStr $urlTitle $cerrmsg]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	515 urllog_log "Error in charset conversion: $urlStr / '$urlTitle': $cerrmsg"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	516 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	517
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	518 # Convert some HTML entities to plaintext and do some cleanup
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	519 set utmp [utl_convert_html_ent $urlTitle]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	520 regsub -all "\r\|\n\|\t" $utmp " " utmp
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	521 regsub -all " *" $utmp " " utmp
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	522 set urlTitle [string trim $utmp]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	523 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	524
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	525 # Check if the URL already exists, just in case we had some redirects
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	526 urllog_add_url $urlStr $urlNick $urlHost $urlChan $urlTitle
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	527 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	528 urllog_verb_msg 1 $urlNick $urlChan "err_http_fail" [list $urlStr $ustatus $uscode $ucode]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	529 urllog_log "Error fetching document: status=$ustatus, code=$ucode, scode=$uscode, url=$urlStr"
83 f171a9fb7b7b urllog: Split urllog_add function to urllog_exists for checking whether given URL already exists in the database. Use urllog_exists where appropriate. Matti Hamalainen <ccr@tnsp.org> parents: 82 diff changeset	530 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	531 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	532 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	533
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	534
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	535 #-------------------------------------------------------------------------
249 d98876dd9ee1 urllog: Rename a function. Matti Hamalainen <ccr@tnsp.org> parents: 241 diff changeset	536 proc urllog_check_line {unick uhost uhand uchan utext} {
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	537 global urllog_log_channels
4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	538
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	539 ### Check the nick
87 97c56d1e9ce2 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 86 diff changeset	540 if {$unick == "*"} {
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	541 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	542 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	543
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	544 ### Check the channel
315 7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	545 if {[utl_match_delim_list $urllog_log_channels $uchan]} {
7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	546 ### Do the URL checking
7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	547 foreach str [split $utext " "] {
464 506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	548 if {[regexp "(\[a-z]+://\[^\[:space:\]\]+\|^(www\|ftp)\.\[^\[:space:\]\]+)" $str ulink]} {
315 7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	549 urllog_check_url $str $unick $uhost $uchan
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	550 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	551 }
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	552 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	553
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	554 return 0
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	555 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	556
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	557
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	558 #-------------------------------------------------------------------------
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	559 ### Parse arguments, find and show the results
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	560 proc urllog_cmd_find {unick uhand uchan utext upublic} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	561 global urllog_db urllog_shorturl urllog_showmax_pub urllog_showmax_priv
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	562
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	563 if {$upublic == 0} {
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	564 set ulimit 5
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	565 } else {
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	566 set ulimit 3
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	567 }
19 9cf22053e5da Repair !urlfind functionality. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	568
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	569 ### Parse the given command
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	570 urllog_log "$unick/$uhand searched URL: $utext"
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	571
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	572 set ftokens [split $utext " "]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	573 set fpatlist ""
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	574 foreach ftoken $ftokens {
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	575 set fprefix [string range $ftoken 0 0]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	576 set fpattern [string range $ftoken 1 end]
295 141bb4a2b76f utillib: utl_escape (which will be deprecated soon). Matti Hamalainen <ccr@tnsp.org> parents: 294 diff changeset	577 set qpattern "'%[utl_escape $fpattern]%'"
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	578
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	579 if {$fprefix == "-"} {
128 0d21b9d1d2b9 urllog: Improve search functionality. Matti Hamalainen <ccr@tnsp.org> parents: 127 diff changeset	580 lappend fpatlist "(url NOT LIKE $qpattern OR title NOT LIKE $qpattern)"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	581 } elseif {$fprefix == "%"} {
128 0d21b9d1d2b9 urllog: Improve search functionality. Matti Hamalainen <ccr@tnsp.org> parents: 127 diff changeset	582 lappend fpatlist "user LIKE $qpattern"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	583 } elseif {$fprefix == "@"} {
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	584 # foo
112 fae3dd7a8b20 urllog: Oops, a typo in variable name. Fixed. Matti Hamalainen <ccr@tnsp.org> parents: 111 diff changeset	585 } elseif {$fprefix == "+"} {
128 0d21b9d1d2b9 urllog: Improve search functionality. Matti Hamalainen <ccr@tnsp.org> parents: 127 diff changeset	586 lappend fpatlist "(url LIKE $qpattern OR title LIKE $qpattern)"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	587 } else {
295 141bb4a2b76f utillib: utl_escape (which will be deprecated soon). Matti Hamalainen <ccr@tnsp.org> parents: 294 diff changeset	588 set qpattern "'%[utl_escape $ftoken]%'"
128 0d21b9d1d2b9 urllog: Improve search functionality. Matti Hamalainen <ccr@tnsp.org> parents: 127 diff changeset	589 lappend fpatlist "(url LIKE $qpattern OR title LIKE $qpattern)"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	590 }
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	591 }
19 9cf22053e5da Repair !urlfind functionality. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	592
27 6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	593 if {[llength $fpatlist] > 0} {
6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	594 set fquery "WHERE [join $fpatlist " AND "]"
6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	595 } else {
6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	596 set fquery ""
6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	597 }
68 3762c621d1c3 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 65 diff changeset	598
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	599 ### Perform SQL query and show results if any
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	600 set nresult 0
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	601 set usql "SELECT id AS uid, utime AS utime, url AS uurl, user AS uuser, host AS uhost, title AS utitle FROM urls $fquery ORDER BY utime DESC LIMIT $ulimit"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	602 urllog_db eval $usql {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	603 incr nresult
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	604 if {[info exists $utitle] && [string length $utitle] > 0} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	605 set stitle [utl_str_map_values [urllog_qm "search_result_title"] [list $utitle [urllog_chop_title $utitle]]]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	606 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	607 set stitle [urllog_qm "search_result_no_title"]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	608 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	609
82 1bbc79f41a1c urllog: Rename few variables for clarity. Matti Hamalainen <ccr@tnsp.org> parents: 81 diff changeset	610 if {$urllog_shorturl != 0 && $uid != ""} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	611 urllog_msg $upublic $unick $uchan "search_result_short" [list $nresult $uuser [utl_ctime $utime] $stitle [urllog_chop_url $uurl] [urllog_get_short $uid]]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	612 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	613 urllog_msg $upublic $unick $uchan "search_result_long" [list $nresult $uuser [utl_ctime $utime] $stitle $uurl]
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	614 }
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	615 }
424 825cac46b1cb Cosmetic / stray trailing whitespace cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 422 diff changeset	616
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	617 if {$nresult == 0} {
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	618 # If no URLs were found
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	619 urllog_msg $upublic $unick $uchan "search_no_match" [list $utext]
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	620 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	621
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	622 return 1
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	623 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	624
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	625
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	626 #-------------------------------------------------------------------------
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	627 ### Finding binded functions
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	628 proc urllog_cmd_pub_find {unick uhost uhand uchan utext} {
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	629 global urllog_search_channels
4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	630
315 7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	631 if {[utl_match_delim_list $urllog_search_channels $uchan]} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	632 return [urllog_cmd_find $unick $uhand $uchan $utext 1]
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	633 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	634 return 0
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	635 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	636
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	637
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	638 proc urllog_cmd_msg_find {unick uhost uhand utext} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	639 return [urllog_cmd_find $unick $uhand "" $utext 0]
3 8003090caa35 Lots of code cleanups, add "fixer" for RasiaTube links (which suck) to point directly to Youtube. Matti Hamalainen <ccr@tnsp.org> parents: 0 diff changeset	640 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	641
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	642
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	643 #-------------------------------------------------------------------------
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	644 # Script initialization
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	645 #-------------------------------------------------------------------------
570 0e3ee1f51c80 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 563 diff changeset	646 ### Initialization messages
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	647 putlog "$urllog_message"
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	648
570 0e3ee1f51c80 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 563 diff changeset	649
0e3ee1f51c80 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 563 diff changeset	650 ### Miscellaneous init messages
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	651 putlog " - Log messages [urllog_is_enabled $urllog_log_enable]"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	652 putlog " - Verbose mode [urllog_is_enabled $urllog_verbose]"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	653 putlog " - Additional URL validity checks [urllog_is_enabled $urllog_extra_checks]"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	654 putlog " - Strict checks [urllog_is_enabled $urllog_extra_strict]"
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	655
570 0e3ee1f51c80 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 563 diff changeset	656
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	657 ### HTTP module initialization
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	658 if {[info exists http_user_agent] && $http_user_agent != ""} {
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	659 ::http::config -useragent $http_user_agent
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	660 } else {
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	661 ::http::config -useragent "$urllog_name/$urllog_version"
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	662 }
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	663
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	664 if {[info exists http_use_proxy] && $http_use_proxy != 0} {
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	665 ::http::config -proxyhost $http_proxy_host -proxyport $http_proxy_port
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	666 putlog " - Using proxy $http_proxy_host:$http_proxy_port"
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	667 }
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	668
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	669 if {[info exists http_tls_support] && $http_tls_support != 0} {
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	670 package require tls
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	671 ::http::register https 443 [list ::tls::socket -request true -require true -ssl2 false -ssl3 false -tls1 true -tls1.1 true -tls1.2 true -cadir $http_tls_cadir -autoservername true]
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	672 putlog " - TLS/SSL support enabled."
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	673 }
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	674
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	675
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	676 ### SQLite database initialization
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	677 if {[catch {sqlite3 urllog_db $urllog_db_file} uerrmsg]} {
561 bdccc83a1c22 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 560 diff changeset	678 putlog "Could not open SQLite3 database '${urllog_db_file}': ${uerrmsg}"
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	679 exit 2
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	680 }
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	681
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	682
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	683 # end of script

Mercurial > hg > egg-tcls

annotate urllog.tcl @ 590:2294b73df2cf