User talk:Paddy3118
RC_vanity_search.py
Sad, but I like to see how my tasks are faring over time:
<lang python> Rosetta Code Vanity search:
How many new pages has someone created?
import urllib, re
user = 'Paddy3118'
site = 'http://www.rosettacode.org' nextpage = site + '/wiki/Special:Contributions/' + user nextpage_re = re.compile(
r'<a href="([^"]+)" title="[^"]+" rel="next">older ')
newpages = [] pagecount = 0 while nextpage:
page = urllib.urlopen(nextpage)
pagecount +=1
nextpage =
for line in page:
if not nextpage:
# Search for URL to next page of results for download
nextpage_match = re.search(nextpage_re, line)
if nextpage_match:
nextpage = (site + nextpage_match.groups()[0]).replace('&', '&')
#print nextpage
npline=line
if '' in line:
# extract N page name from title
newpages.append(line.partition(' title="')[2].partition('"')[0])
page.close()
nontalk = [p for p in newpages if not ':' in p]
print "User: %s has created %i new pages of which %i were not Talk: pages, from approx %i edits" % (
user, len(newpages), len(nontalk), pagecount*50 )
print "New pages created, in order, are:\n ", print "\n ".join(nontalk[::-1])
nextpage = site + '/w/index.php?title=Special:PopularPages'
nextpage_re = re.compile(
r'<a href="([^"]+)" class="mw-nextlink">next ')
data_re = re.compile(
r'^
Sample output on 21:28, 4 June 2009
User: Paddy3118 has created 52 new pages of which 27 were not Talk: pages, from approx 500 edits New pages created, in order, are: Spiral Monty Hall simulation Web Scraping Sequence of Non-squares Anagrams Max Licenses In Use One dimensional cellular automata Conway's Game of Life Data Munging Data Munging 2 Column Aligner Probabilistic Choice Knapsack Problem Yuletide Holiday Common number base conversions Octal Integer literals Command Line Interpreter First-class functions Y combinator Functional Composition Exceptions Through Nested Calls Look-and-say sequence Mutual Recursion Bulls and Cows Testing a Function Select Highest page Ranks for user pages: RANK VIEWS TITLE 102 2442 Monty Hall simulation 106 2294 Knapsack Problem 109 2234 Conway's Game of Life 141 1798 Anagrams 214 1131 Web Scraping 218 1087 Max Licenses In Use 230 1022 Spiral 231 997 One dimensional cellular automata 257 825 Sequence of Non-squares 258 823 Yuletide Holiday 274 762 Column Aligner 314 645 Data Munging 2 318 627 Data Munging 320 623 Probabilistic Choice 322 620 Y combinator 323 614 First-class functions 374 494 Command Line Interpreter 385 446 Functional Composition 403 412 Integer literals 412 404 Mutual Recursion 417 388 Bulls and Cows 438 336 Look-and-say sequence 439 336 Common number base conversions 450 293 Octal 468 250 Exceptions Through Nested Calls 661 75 Select 677 56 Testing a Function >>>
From which I deduce their must be a lot of students out there being set a knapsack like problem ;-)
Sample output on 21:24, 17 June 2009 (UTC)
User: Paddy3118 has created 57 new pages of which 29 were not Talk: pages, from approx 550 edits New pages created, in order, are: Spiral Monty Hall simulation Web Scraping Sequence of Non-squares Anagrams Max Licenses In Use One dimensional cellular automata Conway's Game of Life Data Munging Data Munging 2 Column Aligner Probabilistic Choice Knapsack Problem Yuletide Holiday Common number base formatting Octal Integer literals Command Line Interpreter First-class functions Y combinator Functional Composition Exceptions Through Nested Calls Look-and-say sequence Mutual Recursion Bulls and Cows Testing a Function Select Sort stability Moving Average Highest page Ranks for user pages: RANK VIEWS TITLE 101 2501 Knapsack Problem 102 2499 Monty Hall simulation 108 2309 Conway's Game of Life 137 1876 Anagrams 201 1283 One dimensional cellular automata 214 1162 Web Scraping 220 1114 Max Licenses In Use 226 1073 Spiral 258 844 Yuletide Holiday 259 842 Sequence of Non-squares 271 804 Column Aligner 297 707 Y combinator 310 685 Data Munging 2 311 683 First-class functions 315 666 Data Munging 320 649 Probabilistic Choice 364 525 Command Line Interpreter 377 487 Mutual Recursion 378 480 Functional Composition 391 456 Bulls and Cows 398 447 Integer literals 425 386 Common number base formatting 439 361 Look-and-say sequence 449 320 Octal 458 288 Exceptions Through Nested Calls 518 184 Sort stability 556 146 Testing a Function 577 122 Select 724 4 Moving Average
So in two weeks I've created two new tasks, and the Knapsack Problem has finally moved to become my top viewed task.
Sample output on 07:06, 25 June 2009 (UTC)
I have added no new pages, but I have broken into the top 100 list of page views!!:
Highest page Ranks for user pages: RANK VIEWS TITLE 98 2595 Knapsack Problem 102 2542 Monty Hall simulation 108 2361 Conway's Game of Life 133 1937 Anagrams 199 1312 One dimensional cellular automata 215 1179 Web Scraping 220 1133 Max Licenses In Use 222 1113 Spiral 257 872 Yuletide Holiday 260 870 Sequence of Non-squares 267 829 Column Aligner 296 739 Y combinator 300 720 First-class functions 311 695 Data Munging 2 316 684 Data Munging 320 664 Probabilistic Choice 355 563 Command Line Interpreter 373 519 Mutual Recursion 376 517 Functional Composition 380 496 Bulls and Cows 394 468 Integer literals 418 421 Common number base formatting 435 378 Look-and-say sequence 450 336 Octal 460 305 Exceptions Through Nested Calls 504 214 Sort stability 543 169 Testing a Function 563 146 Select 582 130 Moving Average
I did like writing the story around that Knapsack task.
Vanity Search Updated
After the 4th of July RC updates, My RC Vanity Searcher needed to be updated due to HTML changes. I decided to add a modification, that shows the movement between the order a page was created in, vs the max views order called +/-.
<lang python> Rosetta Code Vanity search:
How many new pages has someone created?
import urllib, re, pdb
user = 'Paddy3118'
site = 'http://www.rosettacode.org' nextpage = site + '/wiki/Special:Contributions/' + user nextpage_re = re.compile(
#r'<a href="([^"]+)" title="[^"]+" rel="next">older ' r'<a href="([^"]+)" title="[^"]+" rel="next"[^>]*>older ' )
newpages = [] pagecount = 0 while nextpage:
page = urllib.urlopen(nextpage)
pagecount +=1
nextpage =
for line in page:
if not nextpage:
# Search for URL to next page of results for download
nextpage_match = re.search(nextpage_re, line)
if nextpage_match:
nextpage = (site + nextpage_match.groups()[0]).replace('&', '&')
#print nextpage
npline=line
if '' in line:
# extract N page name from title
newpages.append(line.partition(' title="')[2].partition('"')[0])
page.close()
nontalk = [p for p in newpages if not ':' in p] nontalk.reverse()
print "User: %s has created %i new pages of which %i were not Talk: pages, from approx %i edits" % (
user, len(newpages), len(nontalk), pagecount*50 )
print "New pages created, in order, are:\n ", print "\n ".join(nontalk)
nextpage = site + '/w/index.php?title=Special:PopularPages'
nextpage_re = re.compile(
#r'<a href="([^"]+)" class="mw-nextlink">next ' r'<a href="([^"]+)"[^>]* class="mw-nextlink"[^>]*>next' )
data_re = re.compile(
r'^Sample output on 12 July 2009
Since the 25th of June I've added two tasks: 'Send an email' and 'Topological (dependency) sort'.
With the new +/- column showing views w.r.t. creation order, you can see that the Knapsack Problem has captured the readers imagination. The Y combinator has a strong showing too.
Is it possible to get views in the last three months rather than total views? That would allow me to discount some of the bias due to date created.
User: Paddy3118 has created 63 new pages of which 31 were not Talk: pages, from approx 600 edits
New pages created, in order, are:
Spiral
Monty Hall simulation
Web Scraping
Sequence of Non-squares
Anagrams
Max Licenses In Use
One dimensional cellular automata
Conway's Game of Life
Data Munging
Data Munging 2
Column Aligner
Probabilistic Choice
Knapsack Problem
Yuletide Holiday
Common number base formatting
Octal
Integer literals
Command Line Interpreter
First-class functions
Y combinator
Functional Composition
Exceptions Through Nested Calls
Look-and-say sequence
Mutual Recursion
Bulls and Cows
Testing a Function
Select
Sort stability
Moving Average
Send an email
Topological (dependency) sort
Highest page Ranks for user pages:
RANK VIEWS +/- TITLE
84 2792 +12 Knapsack Problem
100 2629 +0 Monty Hall simulation
107 2431 +5 Conway's Game of Life
132 2013 +1 Anagrams
199 1368 +2 One dimensional cellular automata
211 1233 -3 Web Scraping
218 1189 -1 Max Licenses In Use
221 1158 -7 Spiral
256 919 -5 Sequence of Non-squares
261 900 +4 Yuletide Holiday
267 866 +0 Column Aligner
281 809 +8 Y combinator
290 780 +6 First-class functions
303 732 -4 Data Munging 2
309 718 -6 Data Munging
315 706 -4 Probabilistic Choice
332 642 +1 Command Line Interpreter
351 598 +6 Mutual Recursion
365 568 +6 Bulls and Cows
366 566 +1 Functional Composition
388 501 -4 Integer literals
407 463 -7 Common number base formatting
418 436 +0 Look-and-say sequence
446 375 -8 Octal
455 347 -3 Exceptions Through Nested Calls
493 248 +2 Sort stability
525 207 -1 Testing a Function
548 186 -1 Select
559 174 +0 Moving Average
703 69 +0 Send an email
725 43 +0 Topological (dependency) sort
Sample output on 24 July 2009
What has happened in the last twelve days?
- Ethiopian Multiplication is newly added and is off to a great start. Editors seem attracted to the challenge. I decided to add this as I was drawn by the simplicity of the algorithm. To make it a better RC task I decided on the restriction that you must create the function using a functions to half, double, and test for even-ness.
- I guess viewers are tired of yet another sort: Topological (dependency) sort still languishes at the bottom.
- I created my first #REDIRECT page: Russian Peasant Multiplication
- Knapsack Problem still continues its rise in ranking (thank you).
- At the top of my table, everything above Sequence of Non-squares has improved its rank. I tried to think of some other thing thats links them, but nothing special comes to mind.
- I am not sure if spiral is unpopular, as it has a lot of views, was my first page ever, but appears seven places down from its original top spot - I remember writing it after creating the Python solution to Zig Zag.
Here's the table:
Highest page Ranks for user pages:
RANK VIEWS +/- TITLE
79 2898 +12 Knapsack Problem
97 2707 +0 Monty Hall simulation
106 2490 +5 Conway's Game of Life
131 2059 +1 Anagrams
196 1391 +2 One dimensional cellular automata
208 1257 -3 Web Scraping
216 1207 -1 Max Licenses In Use
220 1182 -7 Spiral
256 936 -5 Sequence of Non-squares
259 915 +4 Yuletide Holiday
269 878 +0 Column Aligner
270 871 +8 Y combinator
285 803 +6 First-class functions
300 744 -4 Data Munging 2
305 733 -6 Data Munging
312 717 -4 Probabilistic Choice
321 678 +1 Command Line Interpreter
331 648 +6 Mutual Recursion
348 614 +6 Bulls and Cows
360 582 +1 Functional Composition
384 525 -4 Integer literals
396 486 -7 Common number base formatting
407 466 +0 Look-and-say sequence
432 410 -8 Octal
450 367 -3 Exceptions Through Nested Calls
479 275 +6 Ethiopian Multiplication
491 255 +1 Sort stability
516 218 -2 Testing a Function
537 200 -2 Select
540 192 -1 Moving Average
669 90 -1 Send an email
676 79 -1 Topological (dependency) sort
99999 0 +0 Russian Peasant Multiplication
Sample output on 6 August 2009
So, what's been moving!
Well I am glad to see that I have to go right down to Sort stability before I see a task that is lower in the rankings than last time. Y combinator has leap-frogged even more of my tasks, from +8 to +11; and Bulls and Cows, and Ethiopian Multiplication are also doing well.
Newly created is First-class Numbers. When reviewing wp:First-class function, There were comments in the talk page about what languages really have FcF. I decided that rather than bring the discussion to RC, I would create a sister task - similar but using numbers and asking examples to compare the numbers solution with the functions solution. I guess people can still get around it, but I hope they follow the spirit of the task.
Highest page Ranks for user pages:
RANK VIEWS +/- TITLE
76 3064 +12 Knapsack Problem
93 2826 +0 Monty Hall simulation
103 2611 +5 Conway's Game of Life
127 2156 +1 Anagrams
195 1442 +2 One dimensional cellular automata
208 1285 -3 Web Scraping
213 1259 -1 Max Licenses In Use
219 1218 -7 Spiral
249 999 +11 Y combinator
259 966 -6 Sequence of Non-squares
261 956 +3 Yuletide Holiday
268 919 -1 Column Aligner
274 861 +6 First-class functions
298 785 -5 Data Munging
301 776 -3 Probabilistic Choice
302 770 -6 Data Munging 2
315 729 +1 Command Line Interpreter
317 721 +7 Bulls and Cows
321 711 +5 Mutual Recursion
350 631 +1 Functional Composition
373 562 -4 Integer literals
393 508 -7 Common number base formatting
398 501 +0 Look-and-say sequence
424 446 -8 Octal
437 427 +7 Ethiopian Multiplication
451 392 -4 Exceptions Through Nested Calls
496 276 +1 Sort stability
499 267 -2 Testing a Function
502 265 +0 Moving Average
529 226 -3 Select
576 160 -1 Send an email
590 146 -1 Topological (dependency) sort
720 25 +1 First-class Numbers
99999 0 -1 Russian Peasant Multiplication
Sample output on 29 August 2009
The big change that warrants its own section below, is that Ethiopian Multiplication is at the head off the list of pages I started, and ranked number three on RC as a whole.
Now if only I can discover the recipe and start some more, equally attractive pages...
Highest page Ranks for user pages:
RANK VIEWS +/- TITLE
3 17327 +31 Ethiopian Multiplication
71 3351 +11 Knapsack Problem
89 2943 -1 Monty Hall simulation
104 2726 +4 Conway's Game of Life
120 2392 +0 Anagrams
191 1547 +1 One dimensional cellular automata
206 1409 -4 Web Scraping
210 1324 -2 Max Licenses In Use
219 1273 -8 Spiral
228 1171 +10 Y combinator
256 1007 +3 Yuletide Holiday
261 999 -8 Sequence of Non-squares
262 990 -2 Column Aligner
272 947 +5 First-class functions
288 852 -6 Data Munging
293 826 +9 Bulls and Cows
294 823 -7 Data Munging 2
301 812 -6 Probabilistic Choice
302 808 -1 Command Line Interpreter
306 789 +4 Mutual Recursion
337 694 +0 Functional Composition
369 623 -5 Integer literals
385 562 -8 Common number base formatting
388 553 -1 Look-and-say sequence
421 480 -9 Octal
440 438 -4 Exceptions Through Nested Calls
465 365 +2 Average/Simple moving average
485 323 +0 Sort stability
487 313 -3 Testing a Function
517 261 -3 Select
518 261 -1 Send an email
559 211 -1 Topological (dependency) sort
620 147 +3 Simple Random Distribution Checker
661 128 +0 First-class Numbers
667 121 +0 Seven-dice from Five-dice
681 105 +1 List Flattening
99999 0 -4 Russian Peasant Multiplication
Sample output on 17 September 2009
Well, Ethiopian Multiplication is now the most popular page after the front page. Scary! (As I will need to live up to it).
I still like the explanation of the knapsack problem after all these months, so its nice to see it doing well.
Highest page Ranks for user pages:
RANK VIEWS +/- TITLE
2 29238 +31 Ethiopian Multiplication
68 3599 +11 Knapsack Problem
81 3186 -1 Monty Hall simulation
93 3008 +4 Conway's Game of Life
117 2534 +0 Anagrams
190 1605 +1 One dimensional cellular automata
201 1507 -4 Web Scraping
214 1367 -2 Max Licenses In Use
220 1319 -8 Spiral
226 1285 +10 Y combinator
254 1085 +3 Yuletide Holiday
256 1058 -1 Column Aligner
258 1046 +6 First-class functions
260 1032 -10 Sequence of Non-squares
278 929 +10 Bulls and Cows
281 915 -7 Data Munging
286 887 +1 Command Line Interpreter
289 864 -8 Data Munging 2
291 859 +5 Mutual Recursion
297 836 -8 Probabilistic Choice
333 747 +0 Functional Composition
360 673 -5 Integer literals
374 625 +0 Look-and-say sequence
375 623 -9 Common number base formatting
406 523 -9 Octal
417 503 -4 Exceptions Through Nested Calls
434 451 +2 Average/Simple moving average
460 358 +0 Sort stability
463 351 -3 Testing a Function
465 346 +0 Send an email
479 310 -4 Select
480 310 -1 Topological (dependency) sort
515 228 +4 List Flattening
518 218 +2 Simple Random Distribution Checker
525 191 +0 Seven-dice from Five-dice
526 191 -2 First-class Numbers
99999 0 -4 Russian Peasant Multiplication
Sample output on 27 September 2009
Ethiopian Multiplication still adds viewers, and First-class Functions has moved up.
Highest page Ranks for user pages:
RANK VIEWS +/- TITLE
2 34045 +31 Ethiopian Multiplication
67 3713 +11 Knapsack Problem
81 3240 -1 Monty Hall simulation
93 3053 +4 Conway's Game of Life
115 2592 +0 Anagrams
190 1628 +1 One dimensional cellular automata
201 1540 -4 Web Scraping
214 1384 -2 Max Licenses In Use
221 1337 -8 Spiral
224 1319 +10 Y combinator
252 1104 +3 Yuletide Holiday
255 1078 +7 First-class functions
256 1078 -2 Column Aligner
259 1046 -10 Sequence of Non-squares
271 992 +10 Bulls and Cows
275 963 -7 Data Munging
283 912 +1 Command Line Interpreter
285 900 +6 Mutual Recursion
288 877 -9 Data Munging 2
298 851 -8 Probabilistic Choice
332 766 +0 Functional Composition
355 689 -5 Integer literals
373 647 +0 Look-and-say sequence
374 641 -9 Common number base formatting
409 533 -9 Octal
416 520 -4 Exceptions Through Nested Calls
425 496 +2 Average/Simple moving average
455 384 -2 Testing a Function
462 368 -1 Sort stability
465 362 +0 Send an email
473 339 +0 Topological (dependency) sort
479 329 +5 List Flattening
480 327 -6 Select
511 255 +2 Simple Random Distribution Checker
521 218 +0 Seven-dice from Five-dice
523 212 -2 First-class Numbers
99999 0 -4 Russian Peasant Multiplication
Poly thanks
Thanks but if you needed you see the example to get it clearer... it means the pseudocode is not too much clear:D More over, implemeting Octave solution, I've discovered it's more convenient to keep the higher power first (in the "vectors"), so likely I am going to rewrite the pseudocode ;) ... --ShinTakezou 16:31, 18 June 2009 (UTC)
Ethiopian Multiplication
Chalk one up to your vanity search; Ethiopian Multiplication has been the single biggest landing page for Sunday and Monday this week. There's been a huge influx of traffic to it via StumbleUpon. --Short Circuit 02:13, 25 August 2009 (UTC)
- Yah!! (I hope the traffic is OK on your servers though)?
- Lets hope some stay for a browse around the whole site... --Paddy3118 03:48, 25 August 2009 (UTC)
- According to the analytics the bounce rate for stumbleupon referrals for the last three days is about 70%. So some did stay to look around. --Mwn3d 18:32, 25 August 2009 (UTC)
It is great that people are staying to look around, but I am puzzled? I can't find what is sending people to the Ethiopian Multiplication page on the Web. Any links? --Paddy3118 18:21, 26 August 2009 (UTC)
- http://www.stumbleupon.com/url/rosettacode.org/wiki/Ethiopian_Multiplication --Mwn3d 18:22, 26 August 2009 (UTC)
- That one mention means so many hits? Wow. I thought only /. and reddit had that kind of affect. --Paddy3118 18:48, 26 August 2009 (UTC)
- Well that link is from a while ago. Someone else must have found it and put a link to that SU link somewhere else. I'm not sure how to figure out where though. --Mwn3d 18:56, 26 August 2009 (UTC)
- It wasn't slashdot, digg or front-page-reddit level. Those would have resulted in tens of thousands of visits over a day. Peak traffic so far was Monday, where Google Analytics claims as much as 7.7 page views per minute. (Which is actually a bit off, as GA won't register raw, json or css page requests, so the requests would have been much more clumped, or bot edits, which it wouldn't see at all.) There were 368% more visits Monday(3,768) than the previous Monday(804). Each visit monday averaged 1.79 pages/visit, while the previous Monday averaged 3.56. These wouldn't be hard loads for most setups, but we've been riding close to the limits of the Slice under bursty loads for quite a while. I've got an offer to be pretty much the sole occupant of a 4-core Xeon box with 16GB of RAM, but It'll be a gradual transition, if it goes through at all; I won't have root, and I'm uncertain about the current admin's reaction time. Otherwise, I need to find an income source for RC. My number one desire would be to sell content-derived books, but selling t-shirts and the like, even at $5 markup each, would easily make the site pay for itself as it currently sits; Five t-shirts a month would do it. The next most likely option is a donation link, but I hate asking for handouts. And I hate banner ads even more. --Short Circuit 21:22, 26 August 2009 (UTC)
- That one mention means so many hits? Wow. I thought only /. and reddit had that kind of affect. --Paddy3118 18:48, 26 August 2009 (UTC)
Um... I have just seen one of the last pages I created jump to the HEAD of my vanity list, and by a huge margin! After seeing the figures (below), I'm gob-smacked!
RANK VIEWS +/- TITLE 22 7261 +31 Ethiopian Multiplication 71 3276 +11 Knapsack Problem 90 2910 -1 Monty Hall simulation 104 2698 +4 Conway's Game of Life 121 2350 +0 Anagrams 197 1486 +1 One dimensional cellular automata 206 1388 -4 Web Scraping 209 1317 -2 Max Licenses In Use 220 1263 -8 Spiral 235 1112 +10 Y combinator 260 994 +3 Yuletide Holiday 261 989 -8 Sequence of Non-squares 262 983 -2 Column Aligner 272 928 +5 First-class functions 288 836 -6 Data Munging 297 814 -6 Data Munging 2 300 803 -5 Probabilistic Choice 304 794 +7 Bulls and Cows 306 783 -1 Command Line Interpreter 310 772 +4 Mutual Recursion 338 682 +0 Functional Composition 371 605 -5 Integer literals 386 554 -8 Common number base formatting 395 533 -1 Look-and-say sequence 418 473 -9 Octal 443 428 -4 Exceptions Through Nested Calls 466 344 +2 Average/Simple moving average 490 298 -2 Testing a Function 492 297 -1 Sort stability 524 249 -3 Select 529 244 -1 Send an email 562 199 -1 Topological (dependency) sort 666 118 +3 Simple Random Distribution Checker 678 103 +0 First-class Numbers 680 101 +0 Seven-dice from Five-dice 685 87 +1 List Flattening 99999 0 -4 Russian Peasant Multiplication
--Paddy3118 04:28, 25 August 2009 (UTC)
Strange?
Ethiopian Multiplication jumps to be the most popular page behind the Main Page, and I can find no evidence of a 'buzz' around it. No blog entries, no group discussions. I would have expected something more? --Paddy3118 06:55, 1 September 2009 (UTC)
- It's all coming through StumbleUpon. I'd like to write a blog post on exactly what I think happened, but it's going to take a while, and I need to look into it more. A simplified explanation would be that a SU user discovered the page, liked it or reviewed it, leading to both his subscribers (and their subscribers) to see it, as well as increasing the page's general ranking in SU's page selection algorithm. This would be made easier to show if I could identify a trend around the phrase "ethiopian multiplication" in blogs and searches over the last month. The functionality is there in Google's advanced search, but it requires time and opportunity. --Short Circuit 14:33, 1 September 2009 (UTC)