egg-tcls: urllog.tcl annotate

annotate urllog.tcl @ 591:b11484cb3623

urllog: Update instructions a bit.

author	Matti Hamalainen <ccr@tnsp.org>
date	Wed, 27 Jan 2021 13:22:46 +0200
parents	2294b73df2cf
children	ac72a228089b

rev	line source
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	1 ##########################################################################
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	2 #
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	3 # URLLog v2.7.0 by Matti 'ccr' Hamalainen <ccr@tnsp.org>
578 14dfb925a64a Bump copyright years. Matti Hamalainen <ccr@tnsp.org> parents: 573 diff changeset	4 # (C) Copyright 2000-2021 Tecnic Software productions (TNSP)
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	5 #
113 077c7383f36f urllog: Add line about the script's license. Matti Hamalainen <ccr@tnsp.org> parents: 112 diff changeset	6 # This script is freely distributable under GNU GPL (version 2) license.
077c7383f36f urllog: Add line about the script's license. Matti Hamalainen <ccr@tnsp.org> parents: 112 diff changeset	7 #
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	8 ##########################################################################
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	9 #
591 b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	10 # URL-logger script for EggDrop IRC robot, utilizing TCL SQLite3 database
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	11 # interface. Requirements for this script are as follows:
458 cfbe6acc1d73 urllog: tcl-tls 1.7.x (tested with 1.7.13) is now a requirement. It is Matti Hamalainen <ccr@tnsp.org> parents: 457 diff changeset	12 #
591 b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	13 # - utillib.tcl (available from same repository as this script)
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	14 # - tcl-tls 1.7.13+ for SSL/TLS support
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	15 # - TCL 8.6 or later
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	16 # - Eggdrop 1.6.20 or later
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	17 # - SQLite3 and TCL bindings for it.
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	18 #
b11484cb3623 urllog: Update instructions a bit. Matti Hamalainen <ccr@tnsp.org> parents: 590 diff changeset	19 # On Debian: tcl8.6 tcl-tls libsqlite3-tcl eggdrop eggdrop-data
50 f69363fc1f61 Update some comments and add a bit of documentation. Matti Hamalainen <ccr@tnsp.org> parents: 49 diff changeset	20 #
479 fc3b6bc37927 urllog: Improve documentation slightly. Matti Hamalainen <ccr@tnsp.org> parents: 473 diff changeset	21 # If you are doing a fresh install, you will need to create the initial
fc3b6bc37927 urllog: Improve documentation slightly. Matti Hamalainen <ccr@tnsp.org> parents: 473 diff changeset	22 # database with the required table schemas. You can do that by running
503 fdd1f0b83685 urllog: Migration from old versions hasn't been supported in reality for a Matti Hamalainen <ccr@tnsp.org> parents: 493 diff changeset	23 # "create_urllog_db.tcl". You also need to set up the configuration in
fdd1f0b83685 urllog: Migration from old versions hasn't been supported in reality for a Matti Hamalainen <ccr@tnsp.org> parents: 493 diff changeset	24 # "config.urllog" file.
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	25 #
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	26 ##########################################################################
13 e06d41fb69d5 Begin work on converting urllog.tcl to use an SQLite3 database instead of flat file. Matti Hamalainen <ccr@tnsp.org> parents: 8 diff changeset	27
263 f01d60175c44 urllog: Move configuration to external file. Matti Hamalainen <ccr@tnsp.org> parents: 260 diff changeset	28 ### The configuration should be in config.urllog in same directory
f01d60175c44 urllog: Move configuration to external file. Matti Hamalainen <ccr@tnsp.org> parents: 260 diff changeset	29 ### as this script. Or change the line below to point where ever
f01d60175c44 urllog: Move configuration to external file. Matti Hamalainen <ccr@tnsp.org> parents: 260 diff changeset	30 ### you wish. See "config.urllog.example" for an example config file.
f01d60175c44 urllog: Move configuration to external file. Matti Hamalainen <ccr@tnsp.org> parents: 260 diff changeset	31 source [file dirname [info script]]/config.urllog
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	32
291 54d34d086b47 urllog: Use the utility lib for entity conversion. Matti Hamalainen <ccr@tnsp.org> parents: 289 diff changeset	33 ### Required utillib.tcl
54d34d086b47 urllog: Use the utility lib for entity conversion. Matti Hamalainen <ccr@tnsp.org> parents: 289 diff changeset	34 source [file dirname [info script]]/utillib.tcl
54d34d086b47 urllog: Use the utility lib for entity conversion. Matti Hamalainen <ccr@tnsp.org> parents: 289 diff changeset	35
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	36
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	37 ##########################################################################
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	38 # No need to look below this line
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	39 ##########################################################################
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	40 package require sqlite3
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	41 package require http
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	42
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	43 set urllog_name "URLLog"
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	44 set urllog_version "2.7.0"
578 14dfb925a64a Bump copyright years. Matti Hamalainen <ccr@tnsp.org> parents: 573 diff changeset	45 set urllog_message "$urllog_name v$urllog_version (C) 2000-2021 ccr/TNSP"
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	46
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	47
424 825cac46b1cb Cosmetic / stray trailing whitespace cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 422 diff changeset	48 set urllog_httprep [split "\@\|%40\|{\|%7B\|}\|%7D\|\[\|%5B\|\]\|%5D" "\|"]
561 bdccc83a1c22 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 560 diff changeset	49 set urllog_shorturl_str "ABCDEFGHIJKLNMOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789"
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	50
102 5425dc418505 urllog: Entity data is now in UTF-8, but TCL source files are interpreted with current system locale, which may not be UTF-8. We must therefore "convert" the entity mapping string to UTF-8 to be certain of TCL's interpretation of its encoding. Matti Hamalainen <ccr@tnsp.org> parents: 101 diff changeset	51
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	52 #-------------------------------------------------------------------------
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	53 ### Utility functions
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	54 proc urllog_log {umsg} {
267 da239a953e24 urllog: Change some setting names, etc. Matti Hamalainen <ccr@tnsp.org> parents: 264 diff changeset	55 global urllog_log_enable urllog_name
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	56
267 da239a953e24 urllog: Change some setting names, etc. Matti Hamalainen <ccr@tnsp.org> parents: 264 diff changeset	57 if {$urllog_log_enable != 0} {
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	58 putlog "$urllog_name: $umsg"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	59 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	60 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	61
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	62
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	63 proc urllog_qm {uid} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	64 global urllog_messages
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	65
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	66 if {[info exists urllog_messages($uid)]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	67 return $urllog_messages($uid)
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	68 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	69 return $uid
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	70 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	71 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	72
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	73
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	74 proc urllog_smsg {apublic anick achan amsg {aargs {}}} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	75 global urllog_preferredmsg urllog_cmd_name
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	76 set amsg [string map [list "@cmd@" $urllog_cmd_name] $amsg]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	77 utl_msg_args $urllog_preferredmsg $apublic $anick $achan $amsg $aargs
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	78 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	79
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	80
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	81 proc urllog_msg {apublic anick achan aid {aargs {}}} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	82 urllog_smsg $apublic $anick $achan [urllog_qm $aid] $aargs
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	83 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	84
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	85
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	86 proc urllog_verb_msg {apublic anick achan aid {aargs {}}} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	87 global urllog_verbose
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	88 if {$urllog_verbose != 0} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	89 urllog_msg $apublic $anick $achan $aid $aargs
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	90 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	91 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	92
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	93
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	94 proc urllog_isnumber {uarg} {
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	95 foreach i [split $uarg {}] {
65 31c8c4f50aa6 urllog: Improve urllog_isnumber function. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	96 if {![string match \[0-9\] $i]} { return 0 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	97 }
65 31c8c4f50aa6 urllog: Improve urllog_isnumber function. Matti Hamalainen <ccr@tnsp.org> parents: 62 diff changeset	98 return 1
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	99 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	100
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	101
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	102 proc urllog_is_enabled {uval} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	103 if {$uval} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	104 return "ON."
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	105 } else {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	106 return "OFF."
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	107 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	108 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	109
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	110
116 4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	111 proc urllog_sanitize_encoding {uencoding} {
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	112 regsub -- "^\[a-z\]\[a-z\]_\[A-Z\]\[A-Z\]\." $uencoding "" uencoding
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	113 set uencoding [string tolower $uencoding]
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	114 regsub -- "^iso-" $uencoding "iso" uencoding
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	115 return $uencoding
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	116 }
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	117
294 2bb9bcfb104a Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 291 diff changeset	118
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	119 proc urllog_get_ss {uindex} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	120 global urllog_shorturl_str
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	121 return [string index $urllog_shorturl_str $uindex]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	122 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	123
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	124
150 52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	125 proc urllog_get_short {utime} {
52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	126 global urllog_shorturl_prefix urllog_shorturl_str
52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	127
52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	128 set ulen [string length $urllog_shorturl_str]
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	129
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	130 set u1 [expr $utime / ($ulen * $ulen)]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	131 set utmp [expr $utime % ($ulen * $ulen)]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	132 set u2 [expr $utmp / $ulen]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	133 set u3 [expr $utmp % $ulen]
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	134
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	135 return "\[ $urllog_shorturl_prefix[urllog_get_ss $u1][urllog_get_ss $u2][urllog_get_ss $u3] \]"
150 52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	136 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	137
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	138
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	139 proc urllog_chop_url {ustr} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	140 global urllog_shorturl_max_orig
68 3762c621d1c3 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 65 diff changeset	141
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	142 if {[string length $ustr] > $urllog_shorturl_max_orig} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	143 return "[string range $ustr 0 $urllog_shorturl_max_orig]..."
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	144 } else {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	145 return $ustr
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	146 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	147 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	148
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	149
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	150 proc urllog_chop_title {ustr} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	151 global urllog_title_max
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	152
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	153 if {[string length $ustr] > $urllog_title_max} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	154 return "[string range $ustr 0 $urllog_title_max]..."
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	155 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	156 return $ustr
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	157 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	158 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	159
241 669842725e2f Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 240 diff changeset	160
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	161 #-------------------------------------------------------------------------
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	162 proc urllog_add_url {urlStr urlNick urlHost urlChan urlTitle} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	163 global urllog_db urllog_shorturl urllog_pub_channels
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	164
18 1e2232135354 More changes for SQLite support. Matti Hamalainen <ccr@tnsp.org> parents: 13 diff changeset	165
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	166 ### Does the URL already exist?
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	167 set usql "SELECT id AS uid, utime AS utime, url AS uurl, user AS uuser, host AS uhost, chan AS uchan, title AS utitle FROM urls WHERE url='[utl_escape $urlStr]'"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	168 urllog_db eval $usql {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	169 urllog_log "URL said by $urlNick ($urlStr) already known"
83 f171a9fb7b7b urllog: Split urllog_add function to urllog_exists for checking whether given URL already exists in the database. Use urllog_exists where appropriate. Matti Hamalainen <ccr@tnsp.org> parents: 82 diff changeset	170
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	171 if {[utl_match_delim_list $urllog_pub_channels $uchan]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	172 if {$urllog_shorturl != 0} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	173 urllog_verb_msg 1 $urlNick $urlChan "url_known_short" [list $uuser $uchan $uhost [utl_ctime $utime] $utitle $uurl [urllog_get_short $uid]]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	174 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	175 urllog_verb_msg 1 $urlNick $urlChan "url_known_long" [list $uuser $uchan $uhost [utl_ctime $utime] $utitle $uurl]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	176 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	177 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	178 return 1
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	179 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	180
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	181 ### Validate title
93 4e02c0219afe urllog: Insert NULL into title column when we didn't get a title. Matti Hamalainen <ccr@tnsp.org> parents: 92 diff changeset	182 if {$urlTitle == ""} {
4e02c0219afe urllog: Insert NULL into title column when we didn't get a title. Matti Hamalainen <ccr@tnsp.org> parents: 92 diff changeset	183 set uins "NULL"
4e02c0219afe urllog: Insert NULL into title column when we didn't get a title. Matti Hamalainen <ccr@tnsp.org> parents: 92 diff changeset	184 } else {
295 141bb4a2b76f utillib: utl_escape (which will be deprecated soon). Matti Hamalainen <ccr@tnsp.org> parents: 294 diff changeset	185 set uins "'[utl_escape $urlTitle]'"
93 4e02c0219afe urllog: Insert NULL into title column when we didn't get a title. Matti Hamalainen <ccr@tnsp.org> parents: 92 diff changeset	186 }
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	187
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	188 ### Attempt to insert into database
295 141bb4a2b76f utillib: utl_escape (which will be deprecated soon). Matti Hamalainen <ccr@tnsp.org> parents: 294 diff changeset	189 set usql "INSERT INTO urls (utime,url,user,host,chan,title) VALUES ([unixtime], '[utl_escape $urlStr]', '[utl_escape $urlNick]', '[utl_escape $urlHost]', '[utl_escape $urlChan]', $uins)"
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	190 if {[catch {urllog_db eval $usql} uerrmsg]} {
83 f171a9fb7b7b urllog: Split urllog_add function to urllog_exists for checking whether given URL already exists in the database. Use urllog_exists where appropriate. Matti Hamalainen <ccr@tnsp.org> parents: 82 diff changeset	191 urllog_log "$uerrmsg on SQL:\n$usql"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	192 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	193 }
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	194 set uid [urllog_db last_insert_rowid]
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	195 urllog_log "Added URL ($urlNick@$urlChan): $urlStr"
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	196
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	197
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	198 ### Let's say something, to confirm that everything went well.
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	199 if {$urllog_shorturl != 0} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	200 set urlShort [urllog_get_short $uid]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	201 set ushort "short"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	202 } else {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	203 set urlShort ""
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	204 set ushort "long"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	205 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	206
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	207 if {[string length $urlTitle] > 0} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	208 set umode "url_added_${ushort}_has_title"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	209 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	210 set umode "url_added_${ushort}_no_title"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	211 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	212
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	213 urllog_verb_msg 1 $urlNick $urlChan $umode [list $urlTitle [urllog_chop_title $urlTitle] $urlStr [urllog_chop_url $urlStr] $urlShort]
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	214 return 1
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	215 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	216
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	217
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	218 #-------------------------------------------------------------------------
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	219 proc urllog_clear_request { urlStatus urlSCode urlCode urlData urlMeta } {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	220
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	221 ### Clear the request data
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	222 upvar $urlStatus ustatus
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	223 upvar $urlSCode uscode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	224 upvar $urlCode ucode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	225 upvar $urlData udata
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	226 upvar $urlMeta umeta
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	227
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	228 unset ustatus
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	229 unset uscode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	230 unset ucode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	231 unset udata
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	232 array unset umeta
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	233 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	234
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	235
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	236 #-------------------------------------------------------------------------
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	237 proc urllog_do_request { urlNick urlChan urlStr urlStatus urlSCode urlCode urlData urlMeta } {
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	238
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	239 upvar $urlStatus ustatus
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	240 upvar $urlSCode uscode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	241 upvar $urlCode ucode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	242 upvar $urlData udata
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	243 upvar $urlMeta umeta
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	244
456 102dc89488af urllog: Improve how http headers are formed. Matti Hamalainen <ccr@tnsp.org> parents: 425 diff changeset	245 set urlHeaders {}
102dc89488af urllog: Improve how http headers are formed. Matti Hamalainen <ccr@tnsp.org> parents: 425 diff changeset	246 lappend urlHeaders "Accept-Encoding" "identity"
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	247 #lappend urlHeaders "Connection" "keep-alive"
456 102dc89488af urllog: Improve how http headers are formed. Matti Hamalainen <ccr@tnsp.org> parents: 425 diff changeset	248
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	249 ### Perform request
456 102dc89488af urllog: Improve how http headers are formed. Matti Hamalainen <ccr@tnsp.org> parents: 425 diff changeset	250 if {[catch {set utoken [::http::geturl $urlStr -timeout 6000 -binary 1 -headers $urlHeaders]} uerrmsg]} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	251 urllog_verb_msg 1 $urlNick $urlChan "err_http_get" [list $urlStr $uerrmsg]
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	252 urllog_log "HTTP request failed: $uerrmsg"
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	253 return 0
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	254 }
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	255
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	256 ### Check status
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	257 set ustatus [::http::status $utoken]
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	258 set uscode [::http::code $utoken]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	259 set ucode [::http::ncode $utoken]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	260
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	261 if {$ustatus != "ok"} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	262 urllog_verb_msg 1 $urlNick $urlChan "err_http_status" [list $urlStr $ustatus $uscode $ucode]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	263 urllog_log "Error in HTTP request: $ustatus / $uscode ($urlStr)"
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	264 return 0
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	265 }
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	266
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	267 ### Get data
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	268 set udata [::http::data $utoken]
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	269 array set umeta [::http::meta $utoken]
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	270 ::http::cleanup $utoken
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	271
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	272 ### Sanitize the metadata KEYS
584 9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	273 foreach {ukey uvalue} [array get umeta] {
9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	274 set ukey [string tolower $ukey]
9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	275 set umeta($ukey) $uvalue
9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	276 }
9b64f201b3a7 urllog: Use lowercase HTTP metadata keys and sanitize them. Matti Hamalainen <ccr@tnsp.org> parents: 583 diff changeset	277
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	278 return 1
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	279 }
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	280
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	281
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	282 #-------------------------------------------------------------------------
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	283 proc urllog_validate_url { urlNick urlChan urlMStr urlMProto urlMHostName } {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	284 global urllog_httprep urllog_shorturl_prefix urllog_shorturl
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	285
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	286 upvar $urlMStr urlStr
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	287 upvar $urlMProto urlProto
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	288 upvar $urlMHostName urlHostName
3 8003090caa35 Lots of code cleanups, add "fixer" for RasiaTube links (which suck) to point directly to Youtube. Matti Hamalainen <ccr@tnsp.org> parents: 0 diff changeset	289
571 d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	290 ### Hack for removing parenthesis around an URL
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	291 if {[regexp {^$(.+)$$} $urlStr urlMatch urlClean]} {
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	292 set urlStr $urlClean
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	293 }
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	294
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	295 if {[regexp {^\[(.+)\]$} $urlStr urlMatch urlClean]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	296 set urlStr $urlClean
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	297 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	298
571 d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	299 ### Clean excess stuff, if any, and attempt to
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	300 ### guess the URL protocol component if it is missing
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	301 if {[regexp "(\[a-z\]+)://\[^ \]+" $urlStr urlMatch urlProto]} {
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	302 set urlStr $urlMatch
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	303 } elseif {[regexp "www\.\[^ \]+" $urlStr urlMatch]} {
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	304 set urlStr "http://$urlMatch"
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	305 } elseif {[regexp "ftp\.\[^ \]+" $urlStr urlMatch]} {
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	306 set urlStr "ftp://$urlMatch"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	307 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	308
95 687bdd74dfac urllog: Check if TLS support is enabled when checking if we can fetch title information via HTTP or SSL/HTTP. Matti Hamalainen <ccr@tnsp.org> parents: 93 diff changeset	309 ### Handle URLs that have an IPv4-address
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	310 if {[regexp "(\[a-z\]+)://(\[0-9\]{1,3})\\.(\[0-9\]{1,3})\\.(\[0-9\]{1,3})\\.(\[0-9\]{1,3})" $urlStr urlMatch urlProto ni1 ni2 ni3 ni4]} {
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	311 # Check if the IP is on local network
572 295c225e3152 urllog: Improve invalid/local IPv4 network check Matti Hamalainen <ccr@tnsp.org> parents: 571 diff changeset	312 if {$ni1 == 127 \|\| $ni1 == 10 \|\| ($ni1 == 192 && $ni2 == 168)} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	313 urllog_verb_msg 1 $urlNick $urlChan "err_url_local_net" [list $urlStr]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	314 urllog_log "URL pointing to local network, ignored (${urlStr})."
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	315 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	316 }
572 295c225e3152 urllog: Improve invalid/local IPv4 network check Matti Hamalainen <ccr@tnsp.org> parents: 571 diff changeset	317 if {$ni1 == 0 \|\| $ni1 >= 255 \|\| $ni2 >= 255 \|\| $ni3 >= 255 \|\| $ni4 >= 255} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	318 urllog_verb_msg 1 $urlNick $urlChan "err_url_invalid_net" [list $urlStr]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	319 urllog_log "URL pointing to invalid network, ignored (${urlStr})."
571 d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	320 return 0
d4d2fda12308 urllog: Improve URL parsing/validation and protocol guessing. Matti Hamalainen <ccr@tnsp.org> parents: 570 diff changeset	321 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	322 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	323
96 e5a6c27be365 urllog: Comments and cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 95 diff changeset	324 ### Check now if we have an ShortURL here ...
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	325 if {[string match "${urllog_shorturl_prefix}*" $urlStr]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	326 urllog_log "Ignoring ShortURL: ${urlStr}"
252 eb2fce89b8ab urllog: Comment out some currently unused code. Matti Hamalainen <ccr@tnsp.org> parents: 251 diff changeset	327 # set uud ""
eb2fce89b8ab urllog: Comment out some currently unused code. Matti Hamalainen <ccr@tnsp.org> parents: 251 diff changeset	328 # set usql "SELECT id AS uid, url AS uurl, user AS uuser, host AS uhost, chan AS uchan, title AS utitle FROM urls WHERE utime=$uud"
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	329 # urllog_db eval $usql {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	330 # urllog_msg 1 $urlNick $urlChan "'$utitle' - $uurl"
252 eb2fce89b8ab urllog: Comment out some currently unused code. Matti Hamalainen <ccr@tnsp.org> parents: 251 diff changeset	331 # return 1
eb2fce89b8ab urllog: Comment out some currently unused code. Matti Hamalainen <ccr@tnsp.org> parents: 251 diff changeset	332 # }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	333 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	334 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	335
95 687bdd74dfac urllog: Check if TLS support is enabled when checking if we can fetch title information via HTTP or SSL/HTTP. Matti Hamalainen <ccr@tnsp.org> parents: 93 diff changeset	336 ### Get URL protocol component
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	337 set urlProto ""
464 506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	338 if {[regexp "(\[a-z\]+)://" $urlStr urlMatch urlProto]} {
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	339 ### Is it a http or ftp url?
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	340 if {$urlProto != "http" && $urlProto != "https" && $urlProto != "ftp"} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	341 urllog_verb_msg 1 $urlNick $urlChan "err_url_proto_class" [list $urlStr $urlProto]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	342 urllog_log "Broken URL: ${urlStr} - unsupported protocol class (${urlProto})."
464 506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	343 return 0
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	344 }
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	345 } else {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	346 urllog_verb_msg 1 $urlNick $urlChan "err_url_proto_no_class" [list $urlStr]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	347 urllog_log "Broken URL: ${urlStr} - no protocol specifier."
464 506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	348 return 0
506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	349 }
95 687bdd74dfac urllog: Check if TLS support is enabled when checking if we can fetch title information via HTTP or SSL/HTTP. Matti Hamalainen <ccr@tnsp.org> parents: 93 diff changeset	350
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	351 ### Check the PORT (if the ":" is there)
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	352 set urlRecord [split $urlStr "/"]
a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	353 set urlHostName [lindex $urlRecord 2]
a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	354 set urlPort [lindex [split $urlHostName ":"] end]
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	355
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	356 if {$urlPort != "" && ![urllog_isnumber $urlPort] && $urlPort != $urlHostName} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	357 urllog_verb_msg 1 $urlNick $urlChan "err_url_invalid_port" [list $urlStr $urlPort]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	358 urllog_log "Broken URL: ${urlStr} - illegal or invalid port '${urlPort}'"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	359 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	360 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	361
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	362
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	363 set urlStr [string map $urllog_httprep $urlStr]
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	364 return 1
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	365 }
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	366
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	367
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	368 #-------------------------------------------------------------------------
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	369 proc urllog_handle_redirect {urlNick urlHost urlChan urlRedirLevel urlProto urlHostName urlStr urlStatus urlSCode urlCode urlData urlMeta} {
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	370
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	371 upvar $urlProto uproto
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	372 upvar $urlHostName uhostname
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	373 upvar $urlStr ustr
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	374 upvar $urlStatus ustatus
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	375 upvar $urlSCode uscode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	376 upvar $urlCode ucode
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	377 upvar $urlData udata
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	378 upvar $urlMeta umeta
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	379
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	380 ### Was result a redirect?
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	381 if {$ucode >= 301 && $ucode <= 303} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	382 ### Check that we have a location header
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	383 if {![info exists umeta(location)]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	384 urllog_verb_msg 1 $urlNick $urlChan "err_redirect_invalid" [list $ustr $ustatus $uscode $ucode $urlRedirLevel]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	385 urllog_log "Invalid redirect without location header: status=$ustatus, code=$ucode, scode=$uscode, url=$ustr, redirLevel=$urlRedirLevel"
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	386 return 0
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	387 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	388
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	389 ### Fix up location URI
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	390 set nustr $umeta(location)
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	391 if {![regexp "\[a-z\]+://" $nustr]} {
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	392 if {[string range $nustr 0 0] != "/"} {
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	393 append nustr "/"
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	394 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	395 set nustr "${uproto}://${uhostname}${nustr}"
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	396 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	397
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	398 ### Validate the target URI
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	399 urllog_log "Redirection #${urlRedirLevel}: $ustr -> $nustr"
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	400 set ustr $nustr
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	401
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	402 if {![urllog_validate_url $urlNick $urlChan ustr uproto uhostname]} {
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	403 return 0
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	404 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	405
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	406 ### Attempt to fetch redirection target
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	407 urllog_clear_request ustatus uscode ucode udata umeta
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	408 if {![urllog_do_request $urlNick $urlChan $ustr ustatus uscode ucode udata umeta]} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	409 urllog_verb_msg 1 $urlNick $urlChan "err_redirect_fail" [list $ustr $ustatus $uscode $ucode $urlRedirLevel]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	410 urllog_log "Error fetching redirect: status=$ustatus, code=$ucode, scode=$uscode, url=$ustr, redirLevel=$urlRedirLevel"
585 a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	411 return 0
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	412 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	413 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	414
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	415 return 1
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	416 }
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	417
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	418
a5dc31f5b44e urllog: Clean up redirection handling and improve error checking and handling. Matti Hamalainen <ccr@tnsp.org> parents: 584 diff changeset	419 #-------------------------------------------------------------------------
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	420 proc urllog_check_url {urlStr urlNick urlHost urlChan} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	421 global urllog_encoding http_tls_support
581 148c7553c50f urllog: Remove rasiatube hack setting, it's not used. Matti Hamalainen <ccr@tnsp.org> parents: 579 diff changeset	422 global urllog_extra_checks urllog_extra_strict
3 8003090caa35 Lots of code cleanups, add "fixer" for RasiaTube links (which suck) to point directly to Youtube. Matti Hamalainen <ccr@tnsp.org> parents: 0 diff changeset	423
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	424 ### Validate URL compoments, etc.
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	425 set urlProto ""
a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	426 set urlHostName ""
a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	427 if {![urllog_validate_url $urlNick $urlChan urlStr urlProto urlHostName]} {
251 e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	428 return 1
e59f0c3ea0f4 urllog: Handle first and second level redirects. Matti Hamalainen <ccr@tnsp.org> parents: 250 diff changeset	429 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	430
267 da239a953e24 urllog: Change some setting names, etc. Matti Hamalainen <ccr@tnsp.org> parents: 264 diff changeset	431 ### Do we perform additional checks?
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	432 if {$urllog_extra_checks == 0 \|\| !(($http_tls_support != 0 && $urlProto == "https") \|\| $urlProto == "http")} {
230 3e3756b113a1 urllog: Cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 229 diff changeset	433 # No optional checks, or it's not http/https.
306 9858b93387a2 urllog: 100L. Matti Hamalainen <ccr@tnsp.org> parents: 304 diff changeset	434 if {$urllog_extra_strict == 0} {
304 f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	435 # Strict checking disabled, so add the URL, if it does not exist already.
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	436 urllog_add_url $urlStr $urlNick $urlHost $urlChan ""
304 f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	437 return 1
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	438 } elseif {$http_tls_support == 0 && $urlProto == "https"} {
304 f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	439 # Strict ENABLED: If TLS support is disabled and we have https, do nothing
f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	440 return 1
327 a5282cdc56e6 urllog: Fix redirection handling for HTTP 1.1. Matti Hamalainen <ccr@tnsp.org> parents: 319 diff changeset	441 } elseif {$urlProto != "http" && $urlProto != "https"} {
304 f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	442 # Strict ENABLED: It's not http, or https
f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	443 return 1
f1589fe20732 urllog: Added urllog_extra_strict option. Matti Hamalainen <ccr@tnsp.org> parents: 302 diff changeset	444 }
7 50b52294e93e urllog: Strip &rlm; entities from titles; Some work on SSL/https support. Matti Hamalainen <ccr@tnsp.org> parents: 4 diff changeset	445
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	446 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	447 ### Does the document pointed by the URL exist?
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	448 if {![urllog_do_request $urlNick $urlChan $urlStr ustatus uscode ucode udata umeta]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	449 return 1
116 4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	450 }
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	451
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	452 ### Handle redirects of 3 levels
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	453 if {![urllog_handle_redirect $urlNick $urlHost $urlChan 1 urlProto urlHostName urlStr ustatus uscode ucode udata umeta]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	454 return 1
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	455 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	456
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	457 if {![urllog_handle_redirect $urlNick $urlHost $urlChan 2 urlProto urlHostName urlStr ustatus uscode ucode udata umeta]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	458 return 1
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	459 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	460
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	461 if {![urllog_handle_redirect $urlNick $urlHost $urlChan 3 urlProto urlHostName urlStr ustatus uscode ucode udata umeta]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	462 return 1
116 4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	463 }
4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	464
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	465 # Final document
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	466 if {$ucode >= 200 && $ucode <= 205} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	467 set uenc_doc ""
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	468 set uenc_http ""
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	469 set uencoding ""
116 4f3edcf72987 urllog: Improvements in document / HTTP encoding handling and conversion. Matti Hamalainen <ccr@tnsp.org> parents: 115 diff changeset	470
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	471 # Get information about specified character encodings
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	472 if {[info exists umeta(content-type)] && [regexp -nocase {charset\s=\s([a-z0-9._-]+)} $umeta(content-type) umatches uenc_http]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	473 # Found character set encoding information in HTTP headers
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	474 }
470 2faf2eb18f26 urllog: Add support for Twitter meta titles. Matti Hamalainen <ccr@tnsp.org> parents: 469 diff changeset	475
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	476 if {[regexp -nocase -- "<meta.\\?content=\"text/html.\\?charset=(\[^\"\])\".\\?/\?>" $udata umatches uenc_doc]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	477 # Found old style HTML meta tag with character set information
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	478 } elseif {[regexp -nocase -- "<meta.\\?charset=\"(\[^\"\])\".\*\?/\?>" $udata umatches uenc_doc]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	479 # Found HTML5 style meta tag with character set information
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	480 }
313 8175ef52889b urllog: Improve URL title functionality. Matti Hamalainen <ccr@tnsp.org> parents: 312 diff changeset	481
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	482 # Make sanitized versions of the encoding strings
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	483 set uenc_http2 [urllog_sanitize_encoding $uenc_http]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	484 set uenc_doc2 [urllog_sanitize_encoding $uenc_doc]
313 8175ef52889b urllog: Improve URL title functionality. Matti Hamalainen <ccr@tnsp.org> parents: 312 diff changeset	485
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	486 # Check if the document has specified encoding
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	487 # KLUDGE!
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	488 set uencoding $uenc_http2
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	489 if {$uencoding == "" && $uenc_doc2 != ""} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	490 set uencoding $uenc_doc2
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	491 } elseif {$uencoding == ""} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	492 # If _NO_ known encoding of any kind, assume the default of iso8859-1
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	493 set uencoding "iso8859-1"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	494 }
150 52350ed97775 urllog: Cleanups, rename/move some global variables. Matti Hamalainen <ccr@tnsp.org> parents: 136 diff changeset	495
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	496 urllog_log "Charsets: http='$uenc_http', doc='$uenc_doc' / sanitized http='$uenc_http2', doc='$uenc_doc2' -> '$uencoding'"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	497
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	498 # Get the document title, if any
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	499 set urlTitle ""
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	500
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	501 if {[regexp -nocase -- "<meta name=\"twitter:title\" content=\"(.\\?)\"\\s\\/\?>" $udata umatches urlTitle]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	502 # ...
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	503 } elseif {[regexp -nocase -- "<title.\\?>(.\\?)</title>" $udata umatches urlTitle]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	504 # ...
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	505 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	506
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	507 # If facebook, get meta info
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	508 if {[regexp -nocase -- "(http\|https):\/\/www.facebook.com" $urlStr]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	509 if {[regexp -nocase -- "<meta name=\"description\" content=\"(.\*\?)\"" $udata umatches urlTmp]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	510 if {$urlTitle != ""} { append urlTitle " :: " }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	511 append urlTitle $urlTmp
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	512 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	513 }
3 8003090caa35 Lots of code cleanups, add "fixer" for RasiaTube links (which suck) to point directly to Youtube. Matti Hamalainen <ccr@tnsp.org> parents: 0 diff changeset	514
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	515 # If character set conversion is required, do it now
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	516 if {$urlTitle != "" && $uencoding != ""} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	517 if {[catch {set urlTitle [encoding convertfrom $uencoding $urlTitle]} cerrmsg]} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	518 urllog_verb_msg 1 $urlNick $urlChan "err_charset" [list $urlStr $urlTitle $cerrmsg]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	519 urllog_log "Error in charset conversion: $urlStr / '$urlTitle': $cerrmsg"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	520 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	521
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	522 # Convert some HTML entities to plaintext and do some cleanup
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	523 set utmp [utl_convert_html_ent $urlTitle]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	524 regsub -all "\r\|\n\|\t" $utmp " " utmp
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	525 regsub -all " *" $utmp " " utmp
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	526 set urlTitle [string trim $utmp]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	527 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	528
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	529 # Check if the URL already exists, just in case we had some redirects
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	530 urllog_add_url $urlStr $urlNick $urlHost $urlChan $urlTitle
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	531 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	532 urllog_verb_msg 1 $urlNick $urlChan "err_http_fail" [list $urlStr $ustatus $uscode $ucode]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	533 urllog_log "Error fetching document: status=$ustatus, code=$ucode, scode=$uscode, url=$urlStr"
83 f171a9fb7b7b urllog: Split urllog_add function to urllog_exists for checking whether given URL already exists in the database. Use urllog_exists where appropriate. Matti Hamalainen <ccr@tnsp.org> parents: 82 diff changeset	534 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	535 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	536 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	537
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	538
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	539 #-------------------------------------------------------------------------
249 d98876dd9ee1 urllog: Rename a function. Matti Hamalainen <ccr@tnsp.org> parents: 241 diff changeset	540 proc urllog_check_line {unick uhost uhand uchan utext} {
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	541 global urllog_log_channels
4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	542
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	543 ### Check the nick
87 97c56d1e9ce2 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 86 diff changeset	544 if {$unick == "*"} {
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	545 return 0
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	546 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	547
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	548 ### Check the channel
315 7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	549 if {[utl_match_delim_list $urllog_log_channels $uchan]} {
7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	550 ### Do the URL checking
7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	551 foreach str [split $utext " "] {
464 506977ea9d0c urllog: Improve URL validation. Matti Hamalainen <ccr@tnsp.org> parents: 458 diff changeset	552 if {[regexp "(\[a-z]+://\[^\[:space:\]\]+\|^(www\|ftp)\.\[^\[:space:\]\]+)" $str ulink]} {
315 7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	553 urllog_check_url $str $unick $uhost $uchan
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	554 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	555 }
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	556 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	557
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	558 return 0
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	559 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	560
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	561
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	562 #-------------------------------------------------------------------------
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	563 ### Parse arguments, find and show the results
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	564 proc urllog_cmd_find {unick uhand uchan utext upublic} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	565 global urllog_db urllog_shorturl urllog_showmax_pub urllog_showmax_priv
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	566
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	567 if {$upublic == 0} {
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	568 set ulimit 5
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	569 } else {
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	570 set ulimit 3
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	571 }
19 9cf22053e5da Repair !urlfind functionality. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	572
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	573 ### Parse the given command
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	574 urllog_log "$unick/$uhand searched URL: $utext"
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	575
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	576 set ftokens [split $utext " "]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	577 set fpatlist ""
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	578 foreach ftoken $ftokens {
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	579 set fprefix [string range $ftoken 0 0]
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	580 set fpattern [string range $ftoken 1 end]
295 141bb4a2b76f utillib: utl_escape (which will be deprecated soon). Matti Hamalainen <ccr@tnsp.org> parents: 294 diff changeset	581 set qpattern "'%[utl_escape $fpattern]%'"
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	582
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	583 if {$fprefix == "-"} {
128 0d21b9d1d2b9 urllog: Improve search functionality. Matti Hamalainen <ccr@tnsp.org> parents: 127 diff changeset	584 lappend fpatlist "(url NOT LIKE $qpattern OR title NOT LIKE $qpattern)"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	585 } elseif {$fprefix == "%"} {
128 0d21b9d1d2b9 urllog: Improve search functionality. Matti Hamalainen <ccr@tnsp.org> parents: 127 diff changeset	586 lappend fpatlist "user LIKE $qpattern"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	587 } elseif {$fprefix == "@"} {
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	588 # foo
112 fae3dd7a8b20 urllog: Oops, a typo in variable name. Fixed. Matti Hamalainen <ccr@tnsp.org> parents: 111 diff changeset	589 } elseif {$fprefix == "+"} {
128 0d21b9d1d2b9 urllog: Improve search functionality. Matti Hamalainen <ccr@tnsp.org> parents: 127 diff changeset	590 lappend fpatlist "(url LIKE $qpattern OR title LIKE $qpattern)"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	591 } else {
295 141bb4a2b76f utillib: utl_escape (which will be deprecated soon). Matti Hamalainen <ccr@tnsp.org> parents: 294 diff changeset	592 set qpattern "'%[utl_escape $ftoken]%'"
128 0d21b9d1d2b9 urllog: Improve search functionality. Matti Hamalainen <ccr@tnsp.org> parents: 127 diff changeset	593 lappend fpatlist "(url LIKE $qpattern OR title LIKE $qpattern)"
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	594 }
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	595 }
19 9cf22053e5da Repair !urlfind functionality. Matti Hamalainen <ccr@tnsp.org> parents: 18 diff changeset	596
27 6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	597 if {[llength $fpatlist] > 0} {
6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	598 set fquery "WHERE [join $fpatlist " AND "]"
6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	599 } else {
6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	600 set fquery ""
6e381916b016 Some fixes in the query mechanisms of QuoteDB and URLLog. Matti Hamalainen <ccr@tnsp.org> parents: 20 diff changeset	601 }
68 3762c621d1c3 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 65 diff changeset	602
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	603 ### Perform SQL query and show results if any
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	604 set nresult 0
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	605 set usql "SELECT id AS uid, utime AS utime, url AS uurl, user AS uuser, host AS uhost, title AS utitle FROM urls $fquery ORDER BY utime DESC LIMIT $ulimit"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	606 urllog_db eval $usql {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	607 incr nresult
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	608 if {[info exists $utitle] && [string length $utitle] > 0} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	609 set stitle [utl_str_map_values [urllog_qm "search_result_title"] [list $utitle [urllog_chop_title $utitle]]]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	610 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	611 set stitle [urllog_qm "search_result_no_title"]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	612 }
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	613
82 1bbc79f41a1c urllog: Rename few variables for clarity. Matti Hamalainen <ccr@tnsp.org> parents: 81 diff changeset	614 if {$urllog_shorturl != 0 && $uid != ""} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	615 urllog_msg $upublic $unick $uchan "search_result_short" [list $nresult $uuser [utl_ctime $utime] $stitle [urllog_chop_url $uurl] [urllog_get_short $uid]]
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	616 } else {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	617 urllog_msg $upublic $unick $uchan "search_result_long" [list $nresult $uuser [utl_ctime $utime] $stitle $uurl]
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	618 }
a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	619 }
424 825cac46b1cb Cosmetic / stray trailing whitespace cleanup. Matti Hamalainen <ccr@tnsp.org> parents: 422 diff changeset	620
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	621 if {$nresult == 0} {
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	622 # If no URLs were found
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	623 urllog_msg $upublic $unick $uchan "search_no_match" [list $utext]
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	624 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	625
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	626 return 1
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	627 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	628
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	629
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	630 #-------------------------------------------------------------------------
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	631 ### Finding binded functions
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	632 proc urllog_cmd_pub_find {unick uhost uhand uchan utext} {
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	633 global urllog_search_channels
4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	634
315 7a987b22a817 urllog: Add new configuration option urllog_msg_channels. Matti Hamalainen <ccr@tnsp.org> parents: 313 diff changeset	635 if {[utl_match_delim_list $urllog_search_channels $uchan]} {
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	636 return [urllog_cmd_find $unick $uhand $uchan $utext 1]
219 4e09bcc48851 urllog: Add settings for specifying channels where URL logging is active, and where !urlfind functionality works (separately, if so desired.) Matti Hamalainen <ccr@tnsp.org> parents: 218 diff changeset	637 }
28 a59e312b1513 Remove tabs and reindent. Matti Hamalainen <ccr@tnsp.org> parents: 27 diff changeset	638 return 0
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	639 }
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	640
1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	641
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	642 proc urllog_cmd_msg_find {unick uhost uhand utext} {
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	643 return [urllog_cmd_find $unick $uhand "" $utext 0]
3 8003090caa35 Lots of code cleanups, add "fixer" for RasiaTube links (which suck) to point directly to Youtube. Matti Hamalainen <ccr@tnsp.org> parents: 0 diff changeset	644 }
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	645
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	646
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	647 #-------------------------------------------------------------------------
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	648 # Script initialization
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	649 #-------------------------------------------------------------------------
570 0e3ee1f51c80 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 563 diff changeset	650 ### Initialization messages
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	651 putlog "$urllog_message"
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	652
570 0e3ee1f51c80 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 563 diff changeset	653
0e3ee1f51c80 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 563 diff changeset	654 ### Miscellaneous init messages
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	655 putlog " - Log messages [urllog_is_enabled $urllog_log_enable]"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	656 putlog " - Verbose mode [urllog_is_enabled $urllog_verbose]"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	657 putlog " - Additional URL validity checks [urllog_is_enabled $urllog_extra_checks]"
2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	658 putlog " - Strict checks [urllog_is_enabled $urllog_extra_strict]"
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	659
570 0e3ee1f51c80 urllog: Cosmetics. Matti Hamalainen <ccr@tnsp.org> parents: 563 diff changeset	660
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	661 ### HTTP module initialization
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	662 if {[info exists http_user_agent] && $http_user_agent != ""} {
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	663 ::http::config -useragent $http_user_agent
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	664 } else {
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	665 ::http::config -useragent "$urllog_name/$urllog_version"
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	666 }
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	667
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	668 if {[info exists http_use_proxy] && $http_use_proxy != 0} {
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	669 ::http::config -proxyhost $http_proxy_host -proxyport $http_proxy_port
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	670 putlog " - Using proxy $http_proxy_host:$http_proxy_port"
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	671 }
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	672
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	673 if {[info exists http_tls_support] && $http_tls_support != 0} {
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	674 package require tls
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	675 ::http::register https 443 [list ::tls::socket -request true -require true -ssl2 false -ssl3 false -tls1 true -tls1.1 true -tls1.2 true -cadir $http_tls_cadir -autoservername true]
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	676 putlog " - TLS/SSL support enabled."
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	677 }
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	678
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	679
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	680 ### SQLite database initialization
590 2294b73df2cf urllog: Massive refactoring to use similar message system as other scripts, Matti Hamalainen <ccr@tnsp.org> parents: 586 diff changeset	681 if {[catch {sqlite3 urllog_db $urllog_db_file} uerrmsg]} {
561 bdccc83a1c22 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 560 diff changeset	682 putlog "Could not open SQLite3 database '${urllog_db_file}': ${uerrmsg}"
560 17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	683 exit 2
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	684 }
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	685
17183d85ab62 urllog: Cleanups. Matti Hamalainen <ccr@tnsp.org> parents: 545 diff changeset	686
0 1c4e2814cd41 Initial import. Matti Hamalainen <ccr@tnsp.org> parents: diff changeset	687 # end of script

Mercurial > hg > egg-tcls

annotate urllog.tcl @ 591:b11484cb3623