“Exploration vs Exploitation” Dilemma of autodidacts

In Reinforcement Learning, there is something known as Exploration vs Exploitation tradeoff. In this post I would be talking about the tradeoff and some personal dilemma as well.

Follow this thread once – https://twitter.com/alienelf/status/1082949406977114112

I think if you are in self learning mode then you must have gone through this. So Let me explain a bit about Exploration vs Exploitation dilemma in Reinforcement learning.

In RL online decision making involves a fundamental choice; exploration, where we gather more information that might lead us to better decisions in the future or exploitation, where we make the optimal choice given information we already have gathered.

I bet you already know what I am talking about. Yes, Exactly! We autodidacts have discovered a plethora of knowledge over internet and we have no idea when to stop exploring.

I have to make an honest confession here that I am yet to read all citations of any research paper.

Also, if you are someone who gets excited about any new thing which comes up then my friend we are on the same boat. Give me a five if you too have tons of tabs open on your browser right now which has stuff you saw over a week or even months ago!

This can be dangerous sometimes because it often leads to imposter syndrome. We keep on reading and the more we read we start thinking that we are probably incompetent because there is something new almost everyday on arxiv or twitter or reddit but we are yet to do something substantial.

Here is the list of a few subjects which I really like and have tried to be up to date in them –

  1. Deep Learning
  2. Distributed Systems

While here is the list of subjects of which I had ordered books and downloaded paper but I am yet to open first page –

  1. Computer Architecture
  2. Compilers
  3. Chaos Theory (Mathematics)
  4. Applied Cryptography
  5. Quantum Computing

You can see the difference between the two lists.

God! This post is kind of exposing my procrastinations and dilemmas. But somehow I am feeling good about this so I will keep on writing.

There are a lot of things which I feel I must read before starting something new. However there are some ideas I have which I must apply before reading more. For some ideas I don’t have resources(lowly coder here).

I don’t mind sharing my ideas because If you can pick them up and make some use of it then I’d be more than happy. One such idea is using Temporal Difference Learning to advance AlphaGo Zero agent. Another Idea is to work on “Zero Shot Segmentation in Clutter”, a computer vision problem. These are some ideas in Artificial Intelligence.

Now how should I overcome these things? Honestly, No idea! I think the best I can do is come up with a series of blog posts about a topic. This is going to be beneficial for you as well as me.

Let me know if you too have these kinds of dilemmas.

Happy Coding! 🙂

Leave a Reply

Fill in your details below or click an icon to log in:

WordPress.com Logo

You are commenting using your WordPress.com account. Log Out /  Change )

Google photo

You are commenting using your Google account. Log Out /  Change )

Twitter picture

You are commenting using your Twitter account. Log Out /  Change )

Facebook photo

You are commenting using your Facebook account. Log Out /  Change )

Connecting to %s