Category Archives: Python

Turning Reddit RSS into a usable thing, or replacing reddit links with article links

Reddit is a great place to follow various news and updates – I’m using it to keep up to date with programming-related stuff, https://www.reddit.com/r/programming/

Although it could be read directly on the website (I guess), I prefer to do it via RSS aggregator (I use Feedly) – it makes it so much easier to track updates. So adding .rss to the Reddit URL, I get all I need in the aggregator (e.g. https://www.reddit.com/r/programming/.rss)

Well, more than I need, actually – it points me to the Reddit page with comments about the article, and those comments are the very last thing I want to see (if you tried to read them at least once, you’d understand; if not – just try; or trust me).

So doing extra navigation by clicking each article’s subject one more time after Reddit page loads, it finally got me – and I wrote a little proxy page that would replace Reddit comment page link with actual article link in the RSS feed XML.

It’s all here: https://bitbucket.org/hydralien/tools/src/f1196e83ebecca1c4e66c49b8f817e1609527cf6/reddit-streamline/?at=master

There’s a bunch of debug stuff there, and Godaddy-caused library loading nonsense, but the gist is simple – could also be tested at http://hydralien.net/py/direddit/r/programming/.rss

Math is beautiful – it’s a sheer magic to me, but now and then some good people publish approachable articles that allow me get a tiny dash of understanding of how things work.

There’s one such article today – https://innovation.vivint.com/introduction-to-reed-solomon-bc264d0794f8 (and complimentary https://medium.com/@jtolds/joseph-louis-lagrange-and-the-polynomials-499cf0742b39) that I could highly recommend. It’s a fairly simplified overview that provides just a basic idea of a particular error correction technique, but it’s simple yet comprehensive.

In fact it was so fascinating I couldn’t stop myself from giving it a (very short and simple) try. I won’t repeat the articles in any way, just post a code with some comments.

Let’s say we have this code in Python (minor comments inline):

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
xs = [ 1.0, # float format to make suure calculation precision is not impacted - it fails badly otherwise
       2.0,
       3.0,
       4.0,
]
 
ys = [ 10, # the values here are random - like in "I made them up". But this is about "any numbers", right?
       42,
       74,
       111,
]
 
# it's a direct representation of what's described in the https://medium.com/@jtolds/joseph-louis-lagrange-and-the-polynomials-499cf0742b39 article
def l0(x):
    return ( (x-xs[1]) * (x-xs[2]) * (x-xs[3]) ) / ( (xs[0] - xs[1]) * (xs[0] - xs[2]) * (xs[0] - xs[3]) )
def l1(x):
    return ( (x-xs[0]) * (x-xs[2]) * (x-xs[3]) ) / ( (xs[1] - xs[0]) * (xs[1] - xs[2]) * (xs[1] - xs[3]) )
def l2(x):
    return ( (x-xs[0]) * (x-xs[1]) * (x-xs[3]) ) / ( (xs[2] - xs[0]) * (xs[2] - xs[1]) * (xs[2] - xs[3]) )
def l3(x):
    return ( (x-xs[0]) * (x-xs[1]) * (x-xs[2]) ) / ( (xs[3] - xs[0]) * (xs[3] - xs[1]) * (xs[3] - xs[2]) )
 
# as well as this
def f(num):
    return ys[0] * l0(num) + ys[1] * l1(num) + ys[2] * l2(num) + ys[3] * l3(num)
 
for x in range(0, 10):
    fx = f(x)
    print "{}: {}".format(x, fx)

And we run it and we get this:

0: -27.0
1: 10.0
2: 42.0
3: 74.0
4: 111.0
5: 158.0
6: 220.0
7: 302.0
8: 409.0
9: 546.0

(you can add more point; you could also use matplotlib to plot them)

So as you can see it reflected the pre-defined 4 points (10, 42, 74 and 111), but also calculated other points on a curve. So let’s say we sent 6 point, but client received only points 1,2,5 and 6 (10, 42, 158, 220).

If we adjust the input values to look like this:

xs = [ 1.0,
       2.0,
       5.0,
       6.0,
]
 
ys = [ 10,
       42,
       158,
       220,
]

and run it again, we’d still get all the values, because 4 values are enough to define the cubic function curve, and these were taken from that very curve:

0: -27.0
1: 10.0
2: 42.0
3: 74.0
4: 111.0
5: 158.0
6: 220.0
7: 302.0
8: 409.0
9: 546.0

Magic, right?

Some extra calculations around it are also available at http://mathonline.wikidot.com/linear-lagrange-interpolating-polynomials

Space Slaves released!

Well, at least these folks:

work_in_progress

are no longer smashing the tripgang.com – not that they did much, but enough to let them off to whatever they fancy under those opaque helmets.

So… it’s been quite a while. Now the aforementioned TripGang hosts these:

  • KML2GPX (convert Google KML mars into GPX or OSMAND-friendly GPX)
  • MapMarks (search and bookmark travel pinpoints)
  • WikiVert (auto-search for all points from WikiTravel-like text/attractions list)

It’s quite curious how things turn out – the initial idea for the resource was completely different (well, who knows, I might get to it some day after all) – but hey, “whatever works”, right? Hopefully these tools (however immature and weak they are) might be useful to someone (and most hopefully to myself).

Well… bon voyage, there’s not much else to say really.

 

Retrieving multi-line sequences from text files

Had some free time and had a need to parse out a number if similar (but not the same) blocks from a log file. There are tools for that – it could be done with a mixture of grep, sed, bash and some arcane magic – but I’m afraid to find the right toolset, learn required keys for each and experiment with their values and combination would take me longer than to just write me a tool. And it was another opportunity to write a few lines in Python, which I don’t do often enough. And I do love text processing.

So I made me a neat little tool that essentially does one simple thing – starts printing the input stream when some (start) trigger is found there, and stops when another (end) occurs. There are some additions – like, print some lines before/after the block, print couns, unique blocks only etc. – but those are glitter, mostly.

Available in a BitBucket repository.

Greatest prime divisor

Just for the record – custom algorithm for searching greatest prime divisor of a number, O(n) worst-case (O(sqrt(n)) taken by prime detection):

def prime(num, get_number=False):
  for divisor in range(2, int(math.sqrt(num)) + 1):
    if num % divisor == 0:
      return False
  return num if get_number else True

def hydra_gpd(num, get_number=None):
  sequence = range(2, int(math.sqrt(num)) + 1)
  gpd = 1
  for divisor in sequence:
    divided = int(num / divisor)
    if num % divisor == 0:
      if prime(divisor) and divisor > gpd:
        gpd = divisor
      if prime(divided) and divided > gpd:
        gpd = divided
      recursive_gpd = hydra_gpd(divided)
      if recursive_gpd > gpd:
        gpd = recursive_gpd
      return gpd
  return gpd

Nothing too special, just to keep it around.