free html hit counter
Posted on: Monday, November 11, 2019 by Rajiv Popat

Why Developers Should Care about GRPC

GRPC has been around for quite some time but it has recently been integrated into .NET Core 3.0 and the tooling support with it is just first class now.

If you write Rest WebAPI / Microservices using .NET Core, you send JSON data over HTTP requests. The service does its work and sends a JSON response back.

Till the time your request object reaches the service it waits and doesn’t begin processing. Then it does it’s work and sends you a response back. Till your browser or client gets the response back fully there is not much the client can do and basically waits. That’s the request-response model we’ve all grown up with.

We’ve had various takes on improving this basic design in the past. GRPC is Google’s take on solving the problem of making RPC calls and leveraging data streams compared to the standard request response model.

Without going into too much theory, GRPC uses Google’s Protocol buffers to generate code which then sends data using specialized streams which happens to be really fast and as the name suggests, allows streaming of both request and response objects.

Streams are better because you can use the data as it comes in. A crude example? Instead of downloading the whole video when you stream a video you can watch the video as it downloads. GRPC uses the same approach for data. If this doesn’t make sense, read on and by the time you’ve mucked around a bit with the code, it will all start making sense.

For this example we’ll use visual studio code. The tooling is much simpler with Visual Studio 2019 but I prefer to use Visual Studio Code as an IDE of choice because it shows me what’s going on under the hood. With visual studio code, I use following plugin for getting proto file syntax highlighting and support directly inside my IDE:

For syntax highlighting you can also use additional plugins like this one:

protoplugin2

I have .NET Core 3 installed on my machine. 

The first thing I do is:

  1. Generate a server project: This is like your Web API that is going to be consumed by the client.
  2. Generate the client project: This is your client that is going to consume the server and get the data by invoking an endpoint/method on the server.

I generate the server-side project using:

grpcserver

The -o specifies the output path and creates a folder called 'server' where the GRPC service is generated.

I reference the following nugets by hopping into the terminal of VS Code:

Dotnet add package GRPC.Net.Client
Dotnet add package Google.Protobuf
Dotnet add package Grpc.Tools

Here are the repositories of these three nugets if you want to know more about them:

GRPC.NET Client.

Google Protocol Buffers

GRPC Tooling. 

Once I've stubbed the code out and added the necessary packages to the project. I build the server using:

Dotnet Build

And then I open the code with VS Code.

grpcprojectserverstructure

Notice the Protos folder? That has the proto files .NET tooling generated for us. Think of the proto files like your WSDL files if you come from a web service world. Proto files are specifications for your service. You write them by hand. You primarily use them to describe your request objects, response objects and your methods. Here is the example of the proto file that I wrote:

protofile

The above proto file basically says:

  1. I have a request object with the “companyName” attribute that is ordered 1 in the list of attributes. This is the request object because I will be passing the company name whose users I want to fetch.
  2. I have a response object with these attributes: userName, firstName, lastName and address. The numbers next to them is the order in which these attributes will be serialized.
  3. I have a method that takes a company name and streams back the list of users to the client. This is indicated by: “rpc GetUserDetails (UserRequest) returns (stream UserResponse);” line of code that you see in the above screenshot.
    The GetUserDetails it the method that accepts a user request and returns a stream of UserResponse. (By default, a stream would be an array of objects that would be streamed to the client).

Every time I add a .proto file I add it to the servers project (.csproj) file:

serverprotofile

Once I’ve done that, I fire the build and Google Tooling nugets generates the c# files for me in the background to actually generate the real request and response classes. With Visual Studio 2019 this tooling is hidden under the hood. With VS code the tooling fires when you build your project using the “Dotnet build” command.

Once I have the stubs I can write the service. In the service, I fetch some hard-coded values from a function. Typically, I would do this fetching from a database/service but for now, let’s keep this simple and focus on GRPC.

Once I fetch the data I just push the data back to the client but instead of sending the data in a response object that is pushed to the client all at once and waiting for the client to "download" the response, I use GRPC to stream the data one user at a time back to the client:

grpcserveractualservice

Typically, I would have just returned the users I get from GetUserFromDb back to the client but that would generate a regular response and I want to stream the users back to the client so I write them asynchronously to the response stream. Also notice the Task.Delay? I do that to simulate any delays that might actually be happening on the server as you process and return each user. This shows that each user that is processed is streamed back to the client even as the server continues it’s processing with additional users.

Each user that I write to the stream now flows back to the client and the client can start doing whatever it wants to do with it rather than waiting for the whole response to complete.

On the client-side, I write a simple .NET Console Application that makes a call to the server. The only thing the client needs to generate code to call the server is a copy of the proto files which contains the specs for the entire service. You would send your proto files to your clients or publish them somewhere.

I copy the same proto files on the client side and include them in my client project as “Client” files. Here is how I modify the project (.csproj) file:

clientsideprojectfile

I modify my client project to include a copy of the same .proto files and then I can fire a build. This generates all the stubs I need on the client-side to call the server.

Once this is done I start writing the client.

clientsidecode

Notice how I am using the Dangerous Accept Any Server Side Certificate Validator in the code above? That’s just for non-production because I am running this without any valid SSL certificate. In your production you would get a real certificate.

See how I am using the while loop to iterate through the response stream? This allows me to get each user from the stream as the server writes to the stream. And once I get the current item from the stream? Well, I am just showing each user on the console as soon as the server processes the user and writes the user object to the stream.

Now when I run the client the client calls the server, starts listening to the stream for response and starts dealing with partial responses as and when these are streamed by the server.

finaloutput

This is cool, because:

  1. The response is streamed over a channel that is much more optimized compared to JSON data being sent over HTTP using Rest. There are posts that seem to suggest that GRPC is 7x to 10x faster than JSON over rest.
  2. I can do the same streaming I did on the response object while receiving data, even when I send data using the request object. So, if you want to send huge data to the server but don't want to wait till the entire data is sent before the server starts processing it, GRPC works for that too. Put simply, it supports two-way streaming.

The post is long, but the actual implementation is tiny and super simple. If you’ve not tried GRPC before I highly recommend downloading the entire sample project I described in this post from here (it’s listed under the HelloGrpc folder) and running the server first and then the client and mucking around with the code.

Given the level at which Visual Studio Code and Visual Studio tooling for GRPC is right now, I personally think it’s really easy to pick up and most Web API developers will benefit from having this additional arrow in their quiver.

If you are a web developer who writes APIs and who cares about performance and payloads, you should care about newer better ways of communication between servers and clients compared to the traditional rest based WebAPIs that send data over JSON.

We moved from XML to JSON because the payloads were smaller in JSON. GRPC is the natural next step for smaller payloads, better compression and two way streaming of data.

Go on, give it a try. It’s super easy and well worth the few minutes you will invest in learning it. Chances are you can put it to good use right away and see huge gains in performance and end-user experience.

posted on Monday, November 11, 2019 1:54:17 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Wednesday, August 21, 2019 by Rajiv Popat

Fixing Gripes And Electric Shocks from Your New Television.

My wife and I are not into television and have not had a television for a very long time. A couple of months we bought our first new television in five years.

Both my wife and I were not sure how much time we would actually spend in front of it and hence the decision to go Froogle. That and I wanted to be able to control my devices and something like a Raspberry Pi is much more customizable than a locked down android television. We settled for a regular, not smart Vu TV and then build our own media center using a Raspberry Pi.

The TV is a 49 inch Grade A+ panel. So the display hardware is not Ultra-HD but on the Full HD side it is as good as it gets. Pretty nice and does what we need it to at a price that's almost half that of a Sony. Very soon we run into a couple of bumps:

My TV is Shocking Me! Strange Electric Shock on the Television Frame:

I initially start with the assumption that this is an issue with my earthing and call an electrician who tells me that my earthing is just fine and the TV is flawed. He asks me to file a request with Vu and walks out.

I look up the forum and discover there are even more expensive TVs out there that have the same issue. I call up support and they replace the TV but the new one has the same issue. This time around they can give me a refund but aren't willing to help with fixing the issue.

Frustrated I end up deciding to put some basic electronics they teach you in class eight to use. The idea is that the TV missing grounding unit that would connect the current flowing with the grounding socket inside the TV but we can do that externally without even opening the TV.

So let's buy a 50 cent copper wire from a local electric shop. Next, we find a spot on the TV that has electric current running through it. In other words, the area that shocks you. You can test this with a regular tester. We tie one end of the copper wire to the frame that has the shock and screw it up securely behind a screw that holds the TV mount.

The other end goes to the earth pin. The  basic idea here is that since the TV does not provide for any grounding or earthing we earth the current running externally on the TV frame directly to the earth pin of the 3 point socket. This works for countries that support 3 pins and the 3rd pin is for grounding.

With the earth pin wound with the copper wire you ensure that any current running through the frame of the television runs through the copper wire, grounding pin and is eventually earthed effectively ensuring you don't get shocked which you touch the TV frame.

That actually works. I touch the TV and no more shocks. I guess Vu is skimping money by not grounding their circuits, but a 50 cent copper wire fixes that. We now have a really simple home made earthing on the TV frame. No more shocks. And the end product looks pretty elegant with the copper wire concealed behind the TV:

The overall result looks pretty neat and you can barely see the copper wire running from the back of the TV frame to the earth socket. Yes, the Pi and the Firestick and all those wires still need to be organized and concealed but the copper wire itself is that tiny green bit you see in the picture. Nothing objectionable.

I've seen a bunch of articles out there about TVs and monitors shocking people but no solution as such and I hope this helps someone who has a similar problem in future. It's simple class eight electronics you might have learned in your physics class put to some basic use.

Strange Skin Tone

The other day my wife and I were watching a stand-up comedy show and the skin tone of characters seems a little… artificial. Turns out this setting is controlled by something called 'Tint' in most TVs and the default Vu settings don't allow end-users to modify tint settings. The tint option is disabled by default, which means you can't change the setting:

When I first see the setting, I realize it's bumped up all the way to 100 and no way to lower it. I wear my nerd glasses and hop on to a special hidden service menu most Vu TV's provide which can be reached by going to the sound setting menu, clicking on sound balance and then typing 1969 on the number pad in your remote. For some reason the folks at Vu like the year on the moon mission and have picked that to open up secret service menus on the TV:

Notice that this mode is pretty powerful and pretty much allows you to control most tiny aspects of the TV a regular user may not even care about.

Once in, you can tell the TV to not use any special intelligence for skin tones by turning off the tint setting (which you are completely allowed to do in this secretly hidden service menu):

I turn the tint down to zero using the special service menu and the problem is gone.

The Backlight is still way too strong:

The backlight of the TV is still too strong and hurts my eyes. Vu doesn't give any option to change that. Even the service menu doesn't have any backlight settings. I panic and think of returning the TV.  But then service menu allows me to change the RGB gain on the LEDs which are all bumped up to 100 by default:

I realize if I bring those down proportionately I can control backlight of my panel. I do just that. The backlight goes much smoother and the strain on my eyes is gone.

Sorted.

The AI Is A Little Too Smart.

AI is the new thing and most TV's want to run the race of adding AI to their picture rendering. Companies like Vu however mostly do a mediocre job at it. The good part is they let you turn this off by disabling noise reduction setting to off.

Much better.

Love-Hate Relationship With My TV

At this moment I have a love-hate relationship with my TV. I love gadgets which I can customize and root into. Most phones I've owned thus far, are rooted. The service menu of Vu essentially gives me root access to the TV and is very powerful. I dig that about the TV.

The fact that I can hack into my TV and have complete control over my TV makes me feel powerful.

The fact that Vu doesn't handle this little gripes out of the box and expects end users to wear their nerd glasses on to fix these issues make me a little annoyed. Meh!

Either way, all my problems with my TV are sorted and I have a two year extended warranty during which I can return the TV if I face any additional issues. So for now, this will have to do.

If you are thinking of buying a Vu TV - here is my honest advice: Buy it only if you are willing to wear your nerd glasses on and do a little bit of tinkering with the TV, both on the hardware and software front. If you are expecting it to work out of the box like a flawless appliance, Vu isn't for you.

Having said that, the TV is a Froogle choice and once you've made the modifications you feel really happy about spending half the money than what you would have spent of other TVs and getting similar picture quality and overall experience.

Couple the dumb TV up with a Firestick TV (which I bought at discount on Amazon) and a Raspberry Pi 3b+ (bought locally) and you'll have a full-blown smart TV with a pretty decent media center, but that's a whole new post in itself.

Being a Nerd Helps.

This post was just about my gripes and issues with Vu TV and how to fix those. If you own a Vu (or any other TV) and are facing similar issues (particularly electric current running through the TV frame), feel free to use some of these fixes and let me know how it goes.

If your TV just works fine, you can still take solace in the fact that even though most of what you buy or download online today is broken it can be fixed with a little bit of tinkering and geeking.

There is value in being a nerd today. You can stop feeling bad about being a geek. Being a geek is no longer a curse. In today's world it is actually a blessing.

Now go fix something that's broken.

posted on Wednesday, August 21, 2019 9:16:26 AM UTC by Rajiv Popat  #    Comments [0]
Posted on: Thursday, August 15, 2019 by Rajiv Popat

The Honest Truth About Independence.

Today, literally the largest democracy of the world celebrates its 73rd independence day and since I am an Indian it seems like an appropriate time to reflect on a fundamental question - What does it mean to be independent?

I'm not talking about independence at a country level but rather, independence at a personal level.

If you've spent any time on YouTube watching the left-leaning Feminist groups and social justice warriors you've probably seen them yelling and screaming about women independence at the top of their voices.

If you've spent any time in the corporate world you've probably seen right-leaning businessmen in suites go on and on about how they are a 'self-made man' and how they are independent.

A jobless student wants her first job because she wants to become independent. The experienced job goer wants to do a startup because he is sick and tired of serving others and wants financial independence.

Independence is a word that gets thrown around a lot these days and the more I hear the word the more I am unsure if the person using the word even understands what the word means.

The bastardized meaning of the word, largely propagated by western media, often conjures up the image of a powerful working woman who is not dependent on anyone to pay her shopping bills. Or a super-rich dude in a suit who can fire anyone he wants in his own kick-ass company. In reality, both of these images are equally stupid images which have nothing to do with independence.

Today, I wanted to throw out some general thoughts about Independence which no-one seems to be speaking about and hopefully start a discussion on the topic of not just what it means to be independent as a country, but moreover what it means to be independent as an individual. In short, I would love to discuss some of the honest but bitter truths about independence that no-one else seems to be talking about.

Here are a few:

#1: Independence doesn't mean you are not dependent on anyone for anything!

The idea that 'independence' means you are not dependent on anyone for anything is by far the stupidest literal translation of the word that most people fall for.

My being independent doesn't mean I'm not dependent on my family and loved ones for moral support, or my employer for my salary or my clients to pay me or my employer for my services.

The whole idea that any form of life is independent is so ludicrous that I'm still surprised people fall for that trap in today's day and age. Of course, the idea sounds romantic and alluring - the hero of a story who can stand all by himself and doesn't need anyone for help!

The idea of complete independence sounds romantic and alluring because anything impossible and nonpractical usually sounds romantic and alluring. But in reality, no life form or collective group of life forms is truly independent. Even Independent countries depend on their allies all the time!

Being independent doesn't mean you are not dependent on anything or anyone. It just means:

  1. You are not forcefully dependent on anyone - You freely choose who you depend on and how much you depend on them. And in turn, they choose if want to provide you the services for which you depend on them. By the same logic, they also get to choose if and how much they depend on you. And in turn, you get to choose if you want to provide the service they are depending on you for. It's a web of intricately and deliberately picked choices which aren't forced, but that doesn't mean these choices don't come with repercussions and price tags. A lot of us again are in romantic love with this idea of more free choice, when the reality of life is, that there are no free choices and most of us can't even handle choices we have. Independence is just the ability to choose who you want to depend on mutually.
  2. You are interdependent - You rely on your family for moral support. They rely on you to be there when they need you. Independence is about interdependence that it is not forced. Notice I didn't say 'interdependence that is free' - because all interdependence comes at a price, which, in a free world, both sides should be happy to pay. In my personal opinion, it is people who are the most scared of creating this interdependence (or taking up the responsibility of paying the price for this interdependence) usually hide behind the pretense that they are 'independent' and are often the neediest, clingiest and whiniest people you will ever meet in your life.

#2: Independence Is Never about an Individual Achievement

The idea that someone can achieve independence individually is nothing more than a media hype which falls back on the ideology that every story must have a hero and hence if you are independent you are the hero who achieved it single-handedly. We do that to our freedom fighters as well.

Any independence story that you dissect will have more unspoken heroes than the names you see in the star caste, screen roll or in front of the cameras.

Your parents had to babysit you and clean your diapers for years before you even stopped shitting your pants. And even now with your so-called independence every time you feel like crap your Mom or Dad is the first number you dial for support. It's easy to talk about your independence in hindsight and forget all the unspoken heroes who contributed to it and are contributing to it even now. It's really hard to look back, pause and give them the due credit they deserve. I suggest you start doing that.

The act of achieving independence is seldom an individual act. Dozens of people have to give up their freedom and sacrifice their independence for one individual to become independent. This number runs in millions when we talk about countries.

So the next time you use words like 'I'm an independent woman!' or 'I'm a self-made man!' spend a little bit of time to reflect - Are you? Really?

Contrary to what you would love to believe, your independence has very little to do to with your achievements and accomplishments. Your independence has to do much more about the people standing behind you than it has to do about you at all.

#3: Independence is Hard to Achieve, Harder to Retain (And Even Harder To Handle):

At an individual level when most people say they are independent they have no clue of what they are talking about. Show me one man who says he is independent and I'll show you a man who had most likely never seen a movie in a movie theatre alone and has never gone on a truly solo trip or vacation. Show me a man who says he is independent and I'll most likely show you a man who can't even enjoy his own company.

Independence doesn't mean you withdraw from others. But Independence is about ending the habit of being needy, clingy, whiny and being emotionally dependent on others for petty things like attention and approval. It is about mature, meaningful coming together of powerful individuals because they like each others company. The same is true when powerful countries decide to become allies.

When you look at it from that perspective, Independence is hard to achieve and you have to keep working at it. Most of us mere mortals aren't independent. Most of us can't even wash our clothes without whining when the washing machine breaks down or cook our food day after day without relying on a restaurant when we are fully aware that the junk we buying at that stupid fast food joint isn't good for us. And then we sit in our cafeterias and talk about how we as individuals are independent and self-made.

Most of us aren't even capable of forming an independent thought. So the next time you talk about independence here is an exercise you can try out: Come up with one independent thought. Not something that builds on something you read in a newspaper or something you heard a self-help guru scream about on YouTube. One independent thought that was formed in your head. Be Honest about this exercise and you'll see how hard independence is. Now keep working on that thought and you'll see how you slowly start getting better at developing independent opinions.

#4: No, You're Not There Yet:

No, you still aren't independent. But the beauty of living in a free country is that you have the freedom to practice the idea of independence as a lifestyle. Not the usual independence where you strive to not depend on anyone but the more mature idea of independence where you willingly and happily pay the price of mutual interdependence and develop gratitude for people you depend on. The question is, can you even handle that kind of independence? Or is even your idea of independence, dependent on the stupid cliché media narratives of what it means to be independent? Honestly, the whole idea of independence has very little to do about being independent in the conventional sense. Just a little something to think about.

And now that you have something to think about for today, If you're an Indian and are reading this today, Happy Independence Day.

posted on Thursday, August 15, 2019 5:57:20 PM UTC by Rajiv Popat  #    Comments [0]
Posted on: Saturday, August 3, 2019 by Rajiv Popat

Writing, Social Media, Openness and Forgiveness

Some of the best writers out there are capable of captivating you with their stories because they are high in openness. Take for example Eat, Pray, Love - where the Elizabeth Gilbert describes her own broken relationship with an articulation that requires nerves of steel. Steve Job's biography exposes parts of his life and a past that is not easy for most people to lay open for public consumption.

Yes, I know Steve Jobs did not write his own biography but he did allow the author unrestrained access to parts of his life and past knowing well that his life would be exposed for the whole wide world. The mere thought of giving unrestrained access of our lives for public consumption would make most our stomach's churn.

Some authors hide being the veil of fiction while others face their ghosts and daemons head on with their writing but the simple reality of the profession of writing is that unless you are high in openness you are not going to be a good writer. People are either going to find you boring or fake.

Now, theoretically, this generation should be high in openness because they spend most of their lives on social media where they lay their life open for the rest of the world to consume. So they should be really high in the traits of openness, right? Actually, the more I observe the way the current generation engages with social media websites and the internet in general the more I am convinced that social media hardly has anything to do with openness. Being open is almost an antonym of being fake; and when you selectively pick the most glamorous parts of your life and then photoshop them and edit them to publish them on Facebook you aren't being open - you are just being fake.

With the way most people engage with the internet, I'm almost starting to believe that the more you engage on Facebook, WhatsApp and other social media platforms the more you loose your traits of openness. I could write essay after essay to illustrate the difference between being fake on social media and being high on the trait of openness, but I thought I might be able make one simple diagram that gets the point across much more articulately.

Let's assume you have a really bad fight with your wife, spouse or significant other. Here is what the flow looks like if you have traits of openness about it:

Take the same fight and introduce an aspect of fakeness and social media into it and here is what the vicious cycle of false pretense looks like:

Openness in writing is all about self awareness. You are mindful of your bad experiences, you contemplate on those lessons, then you engage in the act of learning and forgiveness about those bad experiences. This is when you finally become capable of writing about the experience articulately. Then you structure your thoughts and write with the idea of passing on a consistent message and a meaningful lesson. And once you do write about it, you let the experience go and you share it with others so that others can learn and grow from it.

Your openness and writing tells everyone that you are just as vulnerable as the rest of them, that you have learned something new about life and that you aren't ashamed of sharing what you have learned. The focus on your writing is not about you - it's about what you've learned. This is why you should never really write about something you are really angry about. Give it a couple of days. Let your head calm down. Structure your thoughts and  when the focus has moved away from 'you' - then write - but not before you have practiced forgiveness and developed a new insight you want to share with everyone.

The Social Media version of openness on the other hand is just about shoving all the dirt under the carpet. Fight with your wife, go on a materialistic vacation to calm her down, take lots of pictures with fake smiles, share those with the world at large to tell the rest of your social circle how happy you are so that someone from your circle feels they aren't as happy as you are; has a fight with their spouse on how they should go on a vacation too only so that they can go on that vacation click the few pictures, publish them on Facebook and upset someone else in their circle. It's literally a viral vicious cycle that benefits no-one other than Facebook or the social media platforms. As a generation we think our lives are open on social media. They aren't. Publishing a few happy pictures or writing a  few spontaneously depressing or angry political opinions on social media isn't openness.

Openness is hard. It demands that you face your ghosts and skeletons head on. That you contemplate on those, you learn something from your experiences and above all you practice forgiveness and then you write about what you've learned, not with the goal of portraying yourself as a hero or a victim but as a regular guy or girl who has grown just an inch and wants to share an idea or a life lesson.

I know you are on Facebook and Twitter. I even know you have huge WhatsApp list and a decently read blog. But that's not the same as having high openness and being a good writer. There is nothing wrong in publishing vacation pictures but the next time you are about to do that, think about how open you really are. Can you write just as articulately and expressively about a ghost from your past or an ugly skeleton in your cupboard? Can you write about how you were a jerk? Can you write about how you messed things up? Can you write about your own scary mental and phycological struggles?

If you can't, you should think twice about publishing selective happy moments  of your life. It's easy to be a hero in your own novel, or a victim or the guy who lives a perfect life with happy pictures but that has nothing to do with openness. Just a little something to think about the next time you get on Facebook or Instagram.

posted on Saturday, August 3, 2019 10:36:10 AM UTC by Rajiv Popat  #    Comments [0]
Posted on: Monday, April 29, 2019 by Rajiv Popat

Moving Forward With Keystone Habits.

Have you ever wondered why your life, literally and physically, moves out of balance when you are going through a stressful time? Your diet is usually the first thing to go out of control. You stop cooking, and even when you eat out you stop taking the right decisions and start picking all the wrong foods.

Your closet, room and home is usually the next thing to go out of control.

Give me your diet journal for the last one month and I can probably tell you how productive or nonproductive your month was. Let me visit your home on a surprise visit and I'll probably tell you exactly which phase of life you're going through.

And then as these little things go out of control they create a cycle of more panic and anxiety in your life, which in turn leads to more things going out of control.

On the other hand, have you also noticed how doing just a few simple things, can have  a profoundly positive impact on your life and your self-confidence? For example, when you consistently wake up early in the morning, work productively (just for a couple of hours a day), when you consistently eat right, play music, workout; go for a run; or do whatever it is that you do; life feels so… awesome and even your work life and productivity begins to improve.

All of us has at least one thing which if we do right and consistently, we feel in control of our lives and when we don't do that one thing… that's usually a sign that we are losing control and going into a state of depressed helplessness.

That one thing can be different for different individuals. For some, it's waking up early. For others it’s making their bed after they wake up. For many others I know, their daily workout happens to be that thing. Everyone is different but most people have at least one thing which creates a perception of control for them. It makes them feel happy, makes them feel like they are in control of their lives and doing that one thing, makes them feel strong.

We are creatures of control. In an uncertain world where most of us have very little control over anything, including our survival, any perception of control that we can create for ourselves makes us feel safer and stronger than folks who are unable to create that perception.

In the larger scheme of universe where you are an insignificant speck of dust, who  could cease to exist because of a simple road accident, an illusion of control makes you believe that what you do matters and as a result of that, you matter.

If you can pick up one habit, and stick to it, even when life throws its shots at you, you are telling yourself that you take active deliberate decisions and your life runs on those decisions; not a random chain of events controlled by mere chance. You are taking a stand against helplessness.  This is why social scientists call these keystone habits.

This is why when you pick up a keystone habit your life starts to change. For example, most smokers who cultivate the habit of running quit smoking. Most folks who start working out or practicing a musical instrument get better at their work life.

You might say that the perception of control is just an illusion, and I might even agree with you, but that perception of control is exactly what allows us to get up in the morning and go to work. That perception of control is what allows survivors to fight dreadful diseases and even though this would sound a little cliché I would argue that this perception is exactly what powers most of our space programs and allows individuals to climb mountains. Of course rockets have exploded in the past and people have died while climbing mountains but that perception and illusion of control is what enables the astronauts and the climbers to roll the dice, take a chance and succeed.

I have a personal theory that as nerds most of us love programming because in an uncertain world, programing computers creates a perception of control and that perception makes us feel strong and empowered. Hey, we can't change everything, but at least we get to control what's behind the screen of that machine.

All my life, one of the most valuable things I’ve learned it is the idea of inching forward slowly and yet consistently using keystone habits and creating a perception of control is your only chance at that forward movement.

This is why every time I start feeling a sense of helplessness, I fall back to some of my keystone habits to regain that perception of control again. I start writing code each day; I start my daily work out session and I come back to this blog and start writing consistently. And I begin reading. These are four keystone habits that help me inch and claw out of dark and depressing times.

And before I know it, I feel empowered to change things and I'm control. I know the control is just a perception but it still works magically when it comes to taking me out of a state of helplessness and making me productive again.

What are your keystone habits? Did you do them today? Did you take a stand against the punches life throws at you, and actively take out time to do something you committed to? If not, please give yourself a moment today to think about which your keystone habits are and then make an active effort to execute on them.

I won’t say it’ll change your life, but if you are experiencing a sense of helplessness, there is a good chance it’ll pull you out of it and empower you to change things. One inch at a time. This is your way of telling yourself that you may not be in control but at least you are not completely out of control either.

Go on. Pick a few keystone habits and give them time, consistently. There is a high chance you may regain your perception of control and life will begin to come back in balance soon. If you haven't given keystone habits a chance, you should. I wish you good luck.

posted on Monday, April 29, 2019 3:41:20 AM UTC by Rajiv Popat  #    Comments [0]
Posted on: Monday, February 4, 2019 by Rajiv Popat

When It Comes To Wellness, You Are Your Scientific Laboratory.

Science is all about cause effect relationships. So when I started working out and set out my first fitness goals a few years ago, I turned to science for validation. I read books like Spark which were all about how workouts can re-wire your brain. If a qualified doctor or researcher shows me a study that involves a sizable number of individuals, control groups, placebos and an output built on causation, I tend to get convinced much more easily about doing something compared to a random individual giving me health and wellness advice. It's probably why I like Headspace; because it approaches Meditation with a scientific approach rather than a random guy telling you that 'meditation is good for you'. It's also the reason why in the last decade I've  literally read dozens if not hundreds of book on neuroscience, health, fitness and wellness. I have a lot of respect for science because it explains 'why' something works.

Having said that, modern day science has it's own set of quirks. One of the primary problems with modern day science, particularly when it comes to wellness, is the more you read, the more contradictions you encounter. As much as I respect science and scientists, (as I grow older) I am starting to realize that If you want to get completely confused about something really simple,  get a bunch of scientists with sufficient funding, contradicting opinions and conflicting interests and ask them to research a topic. Once you've done that, you can sit back and watch in awe as they come back with their elaborate research results with contradicting conclusions and confuse the crap out of you. We like to believe that the years of experience and expertise in a topic makes doctors and scientists better than the rest of us, when in reality, Experts are just as human and are affected by the same biases as the rest of us.

Take for instance the vegan diet. The china study was one of the most detailed elaborate scientific study about how the vegan diet works and heals your body. Just when you thought the science was clear about vegan diet, studies on carnivorous diets started evolving. And this isn't a single isolated example of scientific contradictions when it comes to health and wellness. Take any single simple nutritional opinion and you will see experts fighting in the battle ring of ideas, everyone with equally compelling arguments and 'scientific research' to support their opinions.

How many bananas should you eat on any given day - more or less? Are cocoa beans really good for you or are they dangerous and toxic? Is running good for you or can it kill you? Is soy a superfood or a cancer causing poison? Are mobile phones useful tools that make you super productive or is the radiation from your cell phone killing you and / or making you impotent? Is high cholesterol really connected to heart issues or is that just a medical hype? If you have cholesterol should you take statin drugs or should you continue to try curing it with lifestyle changes? Try googling  any of these supposedly simple questions that you would think science would have figured the answers to by now. Instead, all science has to offer to you in most of these cases is complete confusion and contradictions.

As far as I am concerned, confusion is fine. I am all for confusion. To an extent, I even love confusion. The problem I have with modern science however, is that scientists are now turning to leveraging our primal fear of death to force their 'opinions' (disguised as 'research') on us. And to be honest, most of the information you see out there today isn't even real science.

Someone from the Vegan community wants to prove a point and so touts Vegan Raw diet as a silver bullet and tells you that if you eat dairy you will die. Someone else from the dairy industry fights back by touting Soy as the most toxic substance and argues that if you eat Soy or drink soy milk you will die. Right in the middle of these two far end opinions are regular pragmatic folks like us, who are being constantly influenced by both sides which are leveraging our primal fear of death to push their opinions, agendas and lifestyles on us.

As I grow older, when it comes to my own health and wellness, I am learning how to take these 'scientific' studies with a grain of salt. I am starting to realize that a study based on one individual (you) which most scientists tout as 'anecdotal' is sometimes just as important for your health as any other scientific study done by an expert. When it comes to your own wellness, anecdotal results of what works and does not work for you is sometimes much more important than a doctor telling you what's good for you. Milk, may or may not be good according to science, but if you are lactose intolerant or it makes you feel crappy, it isn't good for you. The fact that milk makes you feel crappy, is reason enough to stop milk even if all the science in the world is telling you milk is harmless or does wonders to your health. On the other hand If you love milk, and it makes you feel energized, continue having milk - don't stop just because the china study says drinking milk is bad for everyone. Your body is your lab, try different things in moderation and see what works and does not work for you.

I had recently developed ulcers in my food pipe which eventually caused some serious gastric issues. I was burping all day long. After going to over four doctors, all of whom had different opinions on what was causing my gastric issues, and after sticking a tube down my throat and conducting a painful endoscopy, I was prescribed a bunch of medicines, which was supposed to fix these ulcers for most patients. All the medicines did however, was temporarily mask the symptoms. Even the masking of symptoms barely lasted for a few weeks after which the symptoms resurfaced again. The moment I would stop the meds I would return to where I was. After going to the same doctor for six months, the symptoms resurfacing every couple of weeks, I went to a new doctor only to be prescribed a different set of tablets.

After a few of these visits someone at work accidently talked to me about infusions made with holy basil and ginger. All you do is you dip them in water like a tea bag and then drink the water. By that time, I had had been prescribed over half a dozen medication by at-least four qualified doctors and nothing had helped; so I was pretty desperate. In an act of desperation I did a quick search on how ginger and holy basil help with gastric issues and bumped into some articles where they had helped individuals with similar symptoms as mine. I started the infusions and within two days, my gastric issues were gone. Placebo? Who knows. Anecdotal? Absolutely. Scientific? Absolutely not. Did it work for me? Hell Yes, it did!

Most of us tend to see our bodies as a machine and if there is anything wrong with it, we walk into a doctors office as if the doctor was a mechanic for the machine. Going by this analogy, most of us know our cars better than we know our own bodies. We can change tires, jump start the engine and we know the kind of gas our car works best on, but we don't seem to know even the basics of our own bodies. How many of us know the power of breathing, regular workouts, meditation, using basic food and herbs as medicine and the science of eating well? How many of us have genuinely experimented with some simple home remedies and herbs to heal ourselves? How about experimenting with different kinds of workouts - from weight training, to cross fit to running, which one works the best for you?

I'm not saying don't go to the doctor for a serious sickness. While your doctor can and will help you with your sickness, it's also equally important that you take control of your own body and start tiny experiments and be mindful about what works for you and what doesn't.

Do you know which foods make you feel like crap? Do you know which foods energize you? Do you know how you can lose some weight or gain some of it back if you really wanted to? Do you know what spikes your blood pressure and what causes it to be normal? Have you tried lowering your cholesterol naturally? Caring about your own wellness is just as much your responsibility as it is your doctor's job.

I'm not saying stop your medications if you have a serious medical condition, but as a parallel step you can start working along with your physician, start some small natural harmless life style changes along with your medication and start observing what works and does not work for you. And as you begin to start using your body as a playground or a lab for your own wellness, you'll get to know yourself better, you'll start having some seriously interesting experiences and over a period of time, you might even start listening to the patterns of your own body which might help you heal yourself and become an exception in a 'scientific study'. When it comes to health, wellness and happiness you are your own scientific lab. You don't have to wait for a bunch of scientists to tell you what to eat or how to do some basic healing - just like you don't need a mechanic for replacing your car tires.

posted on Monday, February 4, 2019 2:59:29 AM UTC by Rajiv Popat  #    Comments [0]
Posted on: Monday, January 7, 2019 by Rajiv Popat

Embracing Controlled Confusion And Flexibility Of Thought.

Back in 2008, about ten years ago, I wrote this blog post where I self righteously proclaimed that when it comes to the time you spend at work, eight hours a day aren't good enough and everyone should spend way more time at their workplace. Like an absolute idiot of the highest grade and kind, who has no life outside of work, I announced:

Eight hours a day might be good enough for a job but they may not be good enough for a profession. And they are absolutely no good for keeping a passion alive. If you can’t love what you do, maybe you need to stop right now.  Look harder for profession you can love. Seriously, if you are going to be an unknown-programmer who writes depressing programmer poetry - it’s best that you chase your dreams in a profession you love.

In just a couple of years, my life took a complete U turn and I was asking programmers to Work Less, Stay Focused And Say No To Random Meaningless Slogging:

Work less, stay focused and if you find yourself moving into a constant firefighting mode for fifteen hours a day and you cannot get shit done, learn how to say no, logout and get some sleep. The same applies for your team if you happen to be leading one.

Couple of years later someone asked a generic question on Reddit - How many hours a day do you work - and I promptly went so far as saying that no-one works for more than 25 hours a week and if you did real work 30 hours a week you would be a rock star:

If your definition of work is producing real and efficient output, the real figure would be more around 25 hours a week.

Let’s be real here - Most work that you do at work is not really work – Filling timesheets, expense-sheets, TPS Reports (:) ) and goofing around with colleagues / friends isn’t work. Reddit isn’t work either. :)

Spend about 30 hours a week “in flow” without gossip, chats, email, twitter and a thousand other beeps and popups distracting you and you should be a star performer at work.

Once you do that for a few years, you can spend more time at work if you love what you do and experience flow but keeping tab of the “number of hours” and just spending time in office to meet that number doesn't make you more productive.

Ask me the same question today and I will tell you that I strongly believe that if anyone did real focused deep work for 15 hours a week, they would be awesome. At 20 hours a week they would probably be top performers in most workplaces.

The early part of this blog endlessly talks about 'passion for work' one post after another. I even wrote childish poems on my passion. Then as I grew older I started questioning what passion really is and my idea of passion completely flipped on it's head. Now I genuinely believe that passion is one of the most overrated things out there.

What's going on here?

Why are my opinions changing in such strikingly contradicting ways?

If you look at the tone and style of my original post on spending more time in office, it's loud, opinionated, assertive and authoritative. Now my tone is much more toned down and inclusive. My opinions are changing over time and so am I. What you see is a classic example of Strong Opinions, Weakly Held. When you give yourself the freedom to change your opinions (even the ones which are really strong) you give yourself the freedom to change your personality over time and evolve as a better person.

I do realize that I sometimes confuse people with constantly evolving opinions. One could argue that if your opinions change so frequently, what is your fixed reality and can your current opinions at any given time be trusted? When you are constantly growing and your opinions are constantly changing evolving, it's easy to confuse yourself as a phony or to give an impression to others that you are clueless... or even worse; that you are actively trying to just prove them wrong by confronting their ideas and opinions because you have a secret vindictive agenda against them.

But then, changing opinions, ability to contemplate, question and challenge both sides of the same idea is what makes us fundamentally human. This is exactly how everything from medicine to space theory has evolved. Doctors for example, know something; till the time there is a breakthrough and then everything they know is wrong and they must unlearn what they know and learn, know and believe something completely contradictory.

In the medical space, every new breakthrough changes everything the doctors know about a particular disease and it's treatment. It doesn't mean that what the doctors knew before was completely useless. It was still saving lives. But the new knowledge and the newly formed approaches to treatment just takes the science of medicine forward and a doctor who is willing to evolve rapidly with the evolving science of medicine is able to help his patients better. There are times for instance when medical science sometimes goes full circle and even starts looking at medicines that were used hundreds of years ago. The doctors who are using science to learn from ancient medicine aren't losers. They are just willing to entertain the thought that maybe they don't have all the answers. It's called research for a reason. You search something, you have an answer, and then you search the answer to the same question again; re-search.

Aristotle once said:

“It is the mark of an educated mind to be able to entertain a thought without accepting it.”

And sometimes when you entertain really good ideas and give them a fair chance, you become a better human being. The process is not about jumping from one idea to another like a monkey. The process is about questioning everything with logic, intellect and with the eye of a philosopher who wants to get to the truth. One philosopher who played this game rather well was Socrates. I find this episode about Socrates particularly amusing:

After his service in the war, Socrates devoted himself to his favorite pastime: the pursuit of truth.

His reputation as a philosopher, literally meaning 'a lover of wisdom', soon spread all over Athens and beyond. When told that the Oracle of Delphi had revealed to one of his friends that Socrates was the wisest man in Athens, he responded not by boasting or celebrating, but by trying to prove the Oracle wrong.

So Socrates decided he would try and find out if anyone knew what was truly worthwhile in life, because anyone who knew that would surely be wiser than him. He set about questioning everyone he could find, but no one could give him a satisfactory answer. Instead they all pretended to know something they clearly did not.

Finally he realized the Oracle might be right after all. He was the wisest man in Athens because he alone was prepared to admit his own ignorance rather than pretend to know something he did not.

Challenging your own opinions and opinions of people around you can be fun. But like all good things it comes at a price. To lead a happy life with constantly evolving opinions, and to embrace the constant questioning of opinions through discourse, you need like minded people around you. People who understand that you are challenging their thoughts and opinions not because you don't 'support' them or you don't 'trust' them; but instead, you are just trying to probe deeper and form your own opinions, ideas and relative reality; so that when you independently come to the conclusions they have come to, you can trust and support them with even more conviction. Sometimes you probe and question ideas and opinions not because you want to fight these ideas and opinions, but because you want to believe them and embrace them with your own independently formed conviction. By questioning and challenging people with your own logic and pragmatic thinking you're trying to make their truth, your truth and make their opinions, your opinions.

With time, I am learning that not everyone wants to understand a philosophy of having strong opinions weakly held. Not everyone wants to dissect opinions they hold strongly and inspect them. There are people who are thin skinned and easily offended. As I grow older I am starting to realize that people often have a fixed reality of life. A fixed map of how they navigate their world and a fixed set of ideas which let them differentiate between good and bad. I am starting to believe that even people who live their life based on a single story should be allowed to live, peacefully.

And if you really want to focus on your intellectual growth and the quest of the truth through logical debate and discourse, without room for feelings or fear of offending someone, you will always find plenty of adventurous people who are open to the idea of fierce logical intellectual arguments and putting their ideas in the intellectual battle ground with yours, match after match to see which ones win and which ones perish, without taking it personally. People who are open to even accepting and embracing your ideas if they win on the merits of logic and depth. People who don't necessarily agree with you all the time; but people who are... your tribe. People who aren't ashamed to learn from you and people who have the capacity to teach you. People who also believe in strong opinions weakly held and the evolution of thought.

The beauty about having documented your thoughts for decades is that you have a live written running documentary on how your brain is evolving. Your writings shows you how you have evolved and changed  as a person over decades. Every-time I go back and read a post or two on my blog, all I find is contradictions. Sometimes the contradictions are so strong that I feel the urge to delete my older posts in an attempt to scrub my stupidity off from public eye; but then, this stupidity and evolution is what makes me... me. It should surprise me, or depress me, or make me feel all confused and phony, but it doesn't. It just makes me happier and open to embracing new stories, ideas, opinions and realities.

Life is so much more than living with just a single story. And if you are the thin skinned kind who wants me to trust you without questioning your opinions or if you want me to not ask questions for the fear of offending people, I won't judge you, but all I will do is make one last humble attempt to try and nudge you towards seeing multiple sides of the same truth and see more than one story of your life. I can't guarantee you success or happiness, but I can tell you that life becomes much more interesting when you do that. Here is to curiosity and adventure of seeking your own truth and then re-seeking it. This year, instead of throwing in emotions and feeling all offended anytime your ideas are challenged, throw your ideas in the ring of logical debate without excessive emotions, see if they win only on the merit of logic, let them go if they don't, embrace new ideas and enjoy some controlled confusion. I promise you, you will see your life become much more colorful. Happy new year.

posted on Monday, January 7, 2019 3:05:33 AM UTC by Rajiv Popat  #    Comments [0]
Posted on: Monday, July 30, 2018 by Rajiv Popat

Mindful Mondays - A Daily Scrum Of Active Questions.

Most of us underestimate the influence of our environments on your daily lives. I've done blog posts where I cite books which tell you how your will power is a depleting resource and  how relying on your will power is a recipe for failure. Triggers, by Marshall Goldsmith is a book that starts with the idea of how the triggers in the environment around us influence our lives and then takes a thought provoking journey through how you can go from triggers constantly and impulsively influencing your life, to you yourself becoming a trigger for positive change in your own life and the life of others you love.

The book suggests multiple ideas and tools to introduce better mindfulness in your life, to avoid mindless drifting and get better control of your life and your behavior. One tool I found particularly interesting while reading the book was the idea of active questions using a daily questionnaire. The author argues that most of the surveys (and even the questions we ask ourselves) are passive and because they are passive they tend to bring about very little behavior change in our lives.

Marshall explains this through his own study:

In the first study, we used three different groups. The first group was a control group that received no training and was asked “before and after” questions on happiness, meaning, building positive relationships, and engagement.

The second group went to a two-hour training session about “engaging yourself” at work and home. This training was followed up every day (for ten working days) with passive questions:

  1. How happy were you today?
  2. How meaningful was your day?
  3. How positive were your relationships with people?
  4. How engaged were you?
The third group went to the same two-hour training session. Their training was followed up every day (for ten working days) with active questions:
  1. Did you do your best to be happy?
  2. Did you do your best to find meaning?
  3. Did you do your best to build positive relationships with people?
  4. Did you do your best to be fully engaged?

At the end of two weeks, the participants in each of the three groups were asked to rate themselves on increased happiness, meaning, positive relationships, and engagement.

The results were amazingly consistent. The control group showed little change (as control groups are wont to do). The passive questions group reported positive improvement in all four areas. The active questions group doubled that improvement on every item! Active questions were twice as effective at delivering training’s desired benefits to employees. While any follow-up was shown to be superior to no follow-up, a simple tweak in the language of follow-up—focusing on what the individual can control—makes a significant difference.

Marshall then goes on to introduce his readers to the idea of active questions that you ask yourself daily. He advices that you appoint a designated coach (could be a relative or a friend) and run through these set of questions with them every night, creating a sense of accountability and mindfulness. You can rate yourself on these  questions on the nightly basis (almost like a quick scrum meeting about your own life) and then do incremental improvements over time. I've been doing this using  a personal quick and dirty survey for myself for a few days and it works. For example here is one set of personal questions from my overall list of questions I ask myself each night:

Of course there are a few more questions for my professional life, work life and relationships too, but you get the idea. Notice the focus here is on 'trying your level best' (though I seem to also provide some value to the outcome, this is not originally the idea presented in the book).

The simple fact that I would be going through this set of questions every night with a loved one and will be answering these questions honestly, provides me with a the much needed nudge to do my best to be able to answer each of these questions positively. Since this is the first time I am doing this, I am focusing on Mini Habbits with a 25 minute single pomodoro session for each of the things that really matter to me.

Having done this exercise I highly recommend it. If nothing else it makes you a little more mindful about your life and where you are spending it. Marshall promotes an old idea, that the planner within you is literally a different person than the doer within you. And it's easy for the planner within you to think up of grand optimistic plans but it's the doer who has to deal with the environmental triggers and fight procrastination.

What I've discovered is that the right questions, asked in the right way can help bring the planner and doer within you in touch with each other and the realities of daily distractions. When I meet my planners expectations, I'm happy. When I don't, at least I am aware of slipping up and am a little bit more mindful the next day. Active questions that you have a relative, friend or loved one (or even yourself) ask you every day can be an extremely powerful tool if you indulge in the exercise every single day. Go ahead, try it out. I've personally tried it and I highly recommend it.

posted on Monday, July 30, 2018 9:41:43 AM UTC by Rajiv Popat  #    Comments [0]