free html hit counter
Posted on: Sunday, June 13, 2021 by Rajiv Popat

An Open Source Vaccine Tracker For CoWin In India.

The Indian government recently rolled out an excellently made website for scheduling vaccines for all Indians. However a lot of private hospitals are following irregular or different times during which they open slots and because of the demand of vaccines the slots often fill up literally in seconds of opening up.

Today if you are looking for slots manually you are at a huge disadvantage compared to folks who might have automated the process.

Here is a quick python script we wrote to continuously poll co-win servers every x seconds and play a music file on your disk as soon as a slot meeting the criteria is found.

thousandtyonecowintracker

I know a lot of bots are available in telegram but most of them are closed source and by the time they notify you, it’s often already too late. This is open source, free, can run on your laptop / desktop (anything that runs python and play sound files, like a raspberry pi, technically should be able to run this), can be configured to run on parameters you like and can notify you instantly by playing music of your choice.

The Github URL contains the script source, Installation Instructions and documentation on all the variables you can customize.

Hope this makes tracking and scheduling vaccines that much easier.

Genuine thanks to the Indian government and Co-win team for keeping the API’s not just open, but easy to understand and really clean so that writing scripts like this one becomes possible.

posted on Sunday, June 13, 2021 6:53:51 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Tuesday, May 19, 2020 by Rajiv Popat

How To Automate Tasks And Other Mundane Stuff - Part 3.

Automating Buy and Sell Alert for Shares On Your Android Phone.

In my last post, I talked about a simple automation using a python API that constantly monitors price of shares I am interested in and tell me when I should buy and sell them. The API, when called returns a simple JSON which tells me which shares or scripts I should buy and which ones I should sell based:

pythonresponse

Now that I have the API up and running, I need to call that using my mobile phone every hour during business hours and get notified if the price of a share changes and falls in the threshold where I need to act.

That way I can stop staring at my phone all the time and only look at share prices when something I want happens in the stock market.

Automation is all about deliberate and mindful attention and mindlessly staring at stock prices every couple of hours is no way to live specially when trading isn't your full time gig.

I've already talked about how I automate stuff on my android phone using Automagic. So let's use automagic to call this API every hour, parse the results and show alerts and notifications.

Automagic allows me to call an API and parse the JSON responses that I get back from the API using custom code. My overall workflow looks like this:

automagicstockpricescriptworkflow

I use a periodic timer to fire every 25 minutes (you can make this as low or high as you want, even run it every minute if you like) during working hours (1o AM to 6 PM). In these runs, I fire an HTTP-Get (or post, depending on the API you are writing; mine uses a Get) and pass appropriate parameters. Here I'm just passing the username since it's a quick API I wrote using python just for myself.

Once I get the response back from the python script on server I can use the script action in automagic to write custom script to parse the response. My script looks like:

automagicstockpriceactualscript

Now all my buy and sell are contained in a single variable call buysellscript which happens to be a list of strings and I can access that variable from anywhere in the workflow.

If you go back to the entire workflow diagram in the beginning of this post,  the first thing I do after this script parsing the results, is check if the buysellscript variable has a list count of more than zero. This variable will have value (or a count / length of the list will be larger than 0) only if I need to buy or sell shares based on my high price, low price and the live market price of the shares I want to monitor.

If the variable has a value, I need to pay attention to my phone and act. So I use a speech output action and make my phone say out loud using google text-to-speech that I need to buy or sell specific scripts. The phone speaking out loud and telling me that I need to buy or sell specific shares not just grabs my attention but is also often a great conversation starter in parties and in office meetings. :)

Then I show the same message using a dialog box. I can even vibrate my phone in a certain way if I want. All of this grabs my attention when a share becomes worth buying or selling. For the rest of the time I can be completely free and blissfully unware of what the stock market is doing.

And just like that my stock trading pretty much runs on auto-pilot. Earlier I would monitor only a couple of stocks at a time, now I can literally set watches on dozens in my very own private database, set-and-forget, trust the workflow to inform me if something important happens to stocks I am monitoring and move on with my life.

I set fair buy and sell values of stocks every couple of days or weekends and update the database. Once I've done that, I stop looking at the stock prices. No Fear Of Missing Out, No constant distractions.

If you invest in stocks, what are you doing to automate or make your investing experience better? What other things grab your attention and make you stare at your mobile every few hours? Have you thought about automating those things?

A lot of people talk about automating a lot of things and some even take automation to extreme but for me the best automations are not just automations that save time, but automations that give you a peace of mind. Automations that assure you that if something you want to know about happens, you'll be informed. You don't need to be glued to your screen or on a app all the time! Automation isn't about becoming obsessed about saving a few minutes but about getting your life back and this by far is one of the really good examples of automation in my life that does exactly that.

posted on Tuesday, May 19, 2020 8:32:47 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Friday, May 15, 2020 by Rajiv Popat

Automating Buy And Sell Alerts for Shares Using Python.

Learning a new programming language is no different than learning a new spoken language. The biggest hurdle to both isn't picking up the language. It's having a lack of opportunity to then use the language in the real world so you become proficient at it.

I've been playing with python for a few months now and the only reason I've been able to hold on to it and become decently acceptable in the language is because there are a lot of opportunities to use python in your personal life. So if your work doesn't demand that you use python you can still put it to use in the real world to solve real problems.

For example, I have a requirement where I set fair value of stocks and buy or sell shares when the prices of a stock go outside this range. That is, if the price of a share falls beyond a certain value I want to know instantly so that I can buy the share and if the price shoots up beyond a certain value I want to be notified so that I can sell the script. The problem is I don't have time to monitor share prices all the time. So I decided to put python to some good use and see how versatile the language is at automating this task.

Building A Price Tracker For Your Portfolio Which Gives Buy/Sell Notifications Using Python:

The first and the biggest step to this automation is to get the current market price of a share / script. There are a lot of paid services that do this but I wanted to get this for free. Python gives me an excellent library called beautiful soup for parsing HTML so the task of screen scraping shouldn't be as daunting as it is in some other languages.

What I discover is that Yahoo Finance shows me the price of a share. To fetch the price of the share all I have to do is hit a URL: https://in.finance.yahoo.com/quote/ + [name of the script whose price I want to fetch]. So, for example: This URL gets me the price of a script called TVSMOTORS in the Indian National stock exchange.

yahoo

The bit highlighted in blue square above (342.35) is the exact value I need to fetch. To fetch that I select that element using google chrome developer tools, pick the specific div and just click copy selector:

yahoostockpricecopyselector

Now we tweak the selector and it basically looks like this:

#quote-header-info > div . My\(6px\) .Pos\(r\). smartphone_Mt\(6px\) > div (without the spaces).

With this I can write a simple script and fetch the price of the script using python using just five lines of code!

stockpricefunction

That's impressive! Screen scarping technologies and frameworks have come a long way and have become really elegant. Now that I can get the price of a script, I can store the high value (above which I want to sell the shares) and low value (price below which I want to buy) in a SQL Server table and then compare those with the real-time prices we fetch using screen scrapping. [Before you flame me for using SQL server with python, I'm doing this because I already have a server with SQL server for this blog :)]. I can do all of that easily in just about 20 lines of code more:

pythongetallresultsforstockprice

That's another 20 lines of code which looks up my SQL table by user, fetches all high and low prices of shares I've preconfigured, compares them with the current price of shares, does this for all shares in my portfolio and tells me if I need to buy or sell specific shares. And I run this on flask:

pythonflaskscriptrunning

And just like that I have an API that can compare of my portfolio of stocks, their low and high prices set by me with the current price in market and then tell me if I should buy or sell specific shares:

JsonResults

I can now host this on any Linux (or windows) box or container and can consume it with a mobile application or even using my automagic automations or a custom mobile application, but those are topics for a different post.

Long story short, in less than 10 lines of code in python I'm able to fetch prices of scripts. Another 40 odd lines and I can compare my entire portfolio and get insights on which stocks I should be buying and selling and turn the whole thing into a WebAPI I can call from anywhere. If I wanted to I can scale this API and open this for everyone because the concept of user is already baked in the design of the database.

In the next couple of posts we will talk about how we can deploy this on a live URL and how we automate the notifications on an android phone. Till then, If you are a .NET developer, and if you've never tried a different language, python is an excellent language to start playing around with because it's really elegant, versatile and with only a few lines of code you can get a lot done. And most of all, you can use it to solve your own real life problems even if your next work project doesn't require you to use python.

posted on Friday, May 15, 2020 5:18:57 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Wednesday, May 13, 2020 by Rajiv Popat

How To Automate Tasks And Other Mundane Stuff - Part 2.

In part 1 we looked at Automagic which allows you a event-logic-action based workflows to automate your mobile and boring and mundane bits of your life. We took an example of creating a simple mindfulness bell.

On the list of things to automate in my life one thing that takes very little time but a lot of psychic weight is picking up, listening to and then hanging up spam calls.

I am on every Do-Not-Call and Do-Not-Disturb registry out there and yet I kept getting a few calls every other day. Everything, ranging from credit card sales call to vacation packages.

There are multiple call blockers you can download and install, but there is a couple of problem with the few I tried:

  1. These folks kept changing their numbers so I wanted to be able to block out patterns rather than blocking individual numbers.
  2. Most of these blockers block the calls without picking them which means the caller doesn't pay for the call. I want to block the spam calls and don't want to get disturbed, but it feels good to know that I made a spammer shell out a tiny bit of money and not get my time or attention in return. :)
  3. When you block calls without picking them and hanging them, the caller still gets an option to leave you a voice mail which can be a nuisance. It's a common problem described here.

With these things I mind I set out to make my own call blocker.

Making a Call Blocker That Supports Pick up and Hang Up in Less than 5 Minutes:

We're going to use automagic for this. If you've gone through my first post on automagic this workflow should look really simple and self explanatory:

automagic-block-call-workflow

The above workflow is really simple. Keep it as a reference as you read the rest of this post.  Like I said automagic works on event - logic and action model. Here, it gives me the event "Incoming call" followed by a few actions. First thing I do is  drag and drop the incoming call event on the workflow.

In the detailed specification of the event (screenshot below), I have the numbers that have 1048909 and 140930 in them getting trapped by the event. The * before and after the number means any other characters or numbers before or after the pattern will still be trapped by this event. I do this since I discovered that all changing numbers from which I receive spam typically start with these numbers.

For you these can be specific numbers or any patterns like these. I even get to pick the state of the incoming call when the event will be fired, where I pick "Ringing", which basically means that I want actions connected to this event to get fired when the phone rings. All of this is set in the detailed view of the event which looks like this:

automagic-receive-phone-event

So when this event happens, i.e. I get a call from any of the these numbers and when my phone rings, I want to fire a couple of actions. These actions are: Answer the ringing call and End The Call.

If you look at the original workflow picture in the beginning of this post, both of these actions follow the incoming call event. Both actions are readily available in Automagic so I just drop them and connect them to the ringing event.

I also want to know that a spam call was blocked, so I put a speech output action (which leverages android text to speech to literally speak out a message). The commas ",,,," are for waiting for a second each, so I basically wait for 4 seconds and then my phone literally says that a "spam call was blocked". That's the last action on the workflow.

And just like that using Automagic we've built an application that folks on this XDA Forum have been asking for sometime now. :)

This is just one example of how we can use some of these powerful frameworks out there to automate some mundane aspects of our lives. If you are interested in more such automations you can watch out for more post in this series of just subscribe to this blog.

posted on Wednesday, May 13, 2020 6:16:43 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Thursday, May 7, 2020 by Rajiv Popat

How To Automate Tasks And Other Mundane Stuff - Part 1.

Your phone is always on. Always connected. And it's always with you. That makes a perfect automation server to automate most things in your life.

There are a couple of great apps out there that let you do crazy automation with your phone.  One of them is Automagic. I love it because it lets me automate most mindfulness and mundane tasks of my life.

automagiconplaystore

It's a paid tool but given the kick ass extendibility and the stuff that you can do with it - it's worth the couple of bucks the developers make you shell out. The price as of this writing is less than a cup of coffee as a decent safe.

(Full disclosure: I don't know or work with anyone from automagic and neither am I getting paid for this post, I'm just a fan of the app because it kicks some serious a$s!)

The other day, for example, I felt that time was slipping by, so I thought I'd make my own mindfulness bell that rings every twenty minutes. If you don't know yet, mindfulness bells are a thing.

There are lots of apps out there, but why buy or install a dedicated app and give it tons of permission when you can make it yourself with automagic?

Building a Mindfulness Bell with Automagic:

Automagic works on the idea of triggers and actions.

Triggers are events that happen in your phone and actions are things you want to do when a trigger happens.

So, for example, if you receive a call and your phone rings, that's an event or a trigger.

Actions are things you want to do when triggers happen. So when your phone rings (event / trigger) check if the call is from a specific number and if it is, hang up without picking the phone; that's an action.

With automagic you drag and drop triggers, logic and actions to create workflows which get fired when you have programmed them to get fired right inside your phone.

For example my workflow for a mindfulness bell looks like this:

mindfullnessnotificationworkflow

In the above workflow we use an event / trigger called the periodic timer which fires at a configurable time interval i.e. ever X seconds/ minutes / hours. That's the first step of the workflow. I click the edit icon on it, and I pick the configuration of that trigger where I specify an interval of 20 minutes (20m):

 automaticmindfullnesstimertrigger

In the above picture I use a period timer event which fires at a configurable interval. I'm telling it to start firing at 8:00 AM every morning, continue till 9:30 PM and fire every 20 minutes (20m). What happens when this trigger is fired will be an action that I add to the trigger. I now drag and drop a sound action and connect it to the timer event using drag and drop features of automagic IDE. Here is what my sound action looks like:

automagicnotificationsoundaction

I just pick one of the pre-existing chime sounds from the (notification_004) and then I start the workflow. This literally takes me less than 5 minutes to design and start running on my phone and just like that, I've introduced a mindfulness bell in my life.

Notice in the overall workflow picture how I've added another vibrate action as well. That ensures that apart from hearing the sound I also get a vibration every twenty minutes.

The Power of Something Like Automagic

Since I started using Automagic I've built a lot of workflows for myself. I am blown away by the volume of triggers and actions is provides. Literally every tiny thing that can happen on your phone shows up as an event. Literally everything you can do with your phone is an action. This is literally like writing scripts for your phone - and that too graphically.

More and more I find myself spending time to design small workflows more and more for things I need to automate and downloading less and less of small tiny apps that do trivial things.

I genuinely think you should give automagic a try. There are a bunch of other such apps that do the same thing. I liked automagic the most because I personally think it's much more powerful than anything out there in terms of collection of events and actions. You might like a different tool, too and that's cool.

Any tool is fine, but the sooner you pick one such app and start automating workflows . It's fun, it's set-and-forget and it lets you focus on things that really matter to you.

This series of posts is going to focus on automating multiple aspects of your life with this single automagic instance always running on my phone. You can keep looking here for most posts on automating the mundane stuff in your life here. Or just subscribe to the entire blog.

posted on Thursday, May 7, 2020 6:04:56 PM UTC by Rajiv Popat  #    Comments [2]
Posted on: Monday, November 25, 2019 by Rajiv Popat

Automating And Being Mindful Of Stuff That Matters - Part 1.

True Automation And Data Collection In Your Life.

Wearables and fitness trackers were virtually non-existent five years ago. The nerds amongst us were using a physical pedometer to track our steps.

pedometer

Fast forward five years and wearables are now a 25 billion dollar market. They are everywhere!
Even though most metric the wearables track hardly mean anything, wearable health trackers have at least proven that people love the idea of monitoring their health. I personally believe that automation and analytics involve more than just wearing a band and tracking your steps or even heart rate. I have talked about my fascination for automation here.

For me effective automation must satisfy  a couple of simple criteria before it can become a part of my life:

True automation is transparent.

It’s one of the biggest reasons why I don’t like wearables. If you have to check your watch five times every day to see how many steps you walked and stare at your heart rate every hour to infer your health from that, I don’t think you’re automating and tracking anything! All you are doing is losing touch with yourself and cultivating obsessions and anxiety.

That’s what the companies who make fitness trackers want you to do, just like social media companies want you to constantly engage on their platforms, your fitness tracker wants you to keep looking at it, dozens of times a day to get validation about your health and wellbeing. No wonder there are times when these devices fail and people get panic attacks.

You shouldn’t be constantly peeking at a watch or an app to get confirmation on how healthy you are. Your health is something you should be mindful of and your body should talk to you. Your health is something you should feel. You should be able to mindfully listen to your body.

Also, step count metric is a bad measure of fitness, the data you collect is hardly analyzed over long term and the automation of collecting that data using a band, is way too obtrusive.

True automation and data collection works silently. Without you even noticing it. You set it and you forget it.

Really, when the big tech giants of the world are collecting your data from your browsing history they aren’t constantly pinging and buzzing you. That is what makes their data collection so effective. It’s so silent, you don't even know it's happening.

When you work on collecting data about yourself, you need to have similar processes in place for automation and data collection. Also, not everyone needs to collect the same data either, which transitions us to our next point.

True Automation Is Personalized.

As a nerd who has run a half marathon and multiple 10k’s I understand that step count is a bad metric and means nothing. For me, the hours I spend working out is a much better metric than the number of steps I walked.

Collecting the number of steps actually messes me up! I see 16000 steps on a pedometer on most evenings and then I silently convince myself that I have done way more walking today than a regular person so I don't need to work out.

It's a lousy metric that is literally detrimental to my cardiovascular health and overall fitness. Every time I wear a band, the band convinces me that I don't need to work out and my workout sessions come down.

For me, simply counting the number of days I worked out in a month is a way better metric than my step count of every day for an entire year. The point? What matters to me, may not matter to you. True automation is personalized.

For example for me commute is a big deal. I like to hack my time and minimize the time I spend commuting to work and back. It’s such a big deal for me that I need to track and analyze that data. If you live close to your workplace and spend ten minutes walking to office, tracking commute might mean nothing for you.

Spam calls are a serious problem for me and I feel the need to automate blocking those because I literally get multiple spam calls a day. You may not be getting any and may not want to automate blocking those.

Similarly, since I moved away from the city my family lives in, the amount of time I spend talking to my parents and family back home is a big deal for me, so I track that.

Things that matter to everyone are different. Automation should be personalized and if you truly want to automate parts of your life, it's about time you put a bit of programming effort on your own customized automation, taking your own data in your own hand and pick up a few tools of automation that work for you.

In this series of posts, I plan on showcasing how I personalize my automation and share some of the tools I use with you. Every tool I use eventually collects data about my activities and the time I spend. I’ll also show you how that data then pools into a centralized database that I own myself, which brings us to our next topic.

Good Automation Doesn’t Work In Isolation

What I eat has an impact on my mood. How much time I spend on the road actually has an impact on how efficient I am at work. How much sound sleep my wife gets has an impact on how many fights we have. :) Tracking an isolated item like the number of steps or heart rate literally means nothing.

When you start bringing a bunch of these random facts in a central database suddenly you start getting insights you never had before.

If you truly want to automate and analyze your life with data, you need to design and own a database of data points from your life that matter to you.

When you own your own data sets and when you design your own automation it makes it that much more easier for you to connect things and write smarter code and analytics to make sense of your data.

And The Point Of This Series Of Posts Is?

The idea I’m trying to share with you is that you need your own personalized automation and a database of data that really matters to you. I’ll be doing a series of posts here where I talk about things I automate and track in my own life.

In this series of posts, I plan on taking you through some simple automation tools and techniques to make you more effective and help you collect and analyze data about yourself and your loved ones.

We will use a bunch of random collection techniques I use and go through some of the fun automation I’ve set up around my life.

As nerds, most of us are excited about automation, machine learning, and data science but most folks learning it don't have any real project to work on it. Why not put it to use to automate and improve your own life?

Through this series of posts, I want to learn from you more than I want to teach you. Please use my techniques and tools if you like them and go build your own automation and intelligence around what matters most to you. Please use the comments generously or drop me an email to let me know the automation you are doing.

Think of this series of posts as nothing more than a nerd mucking around and having fun with some data and some code. And in the process, I hope to learn and share something meaningful and something useful with you.

In the next post we’ll start with reducing your physic weight and using basic automation on your phone to get things that bother you out of your life. So watch out for this series of posts (or subscribe to this blog) for more on the topic of basic automation, machine learning and analytics to improve your life!

posted on Monday, November 25, 2019 5:53:04 PM UTC by Rajiv Popat  #    Comments [0]