Back Propagation in Neural Networks


Once you understand Feed Forward mechanism which isn't much useful when you do not back propagate to update the weights and refine your Neural Network, your Neural Network isn't of much use. So in this blog post, let us understand what is back propagation and how could we represent the back propagation mathematically! Back propagation generally means that you distribute the errors to the hidden layers in a proportionate manner relative to how much the weights were contributing to the actual error in the subsequent layer. so the larger the weight, the more of the output error is carried back to the hidden layer.

As always, here is a sheet of paper where I have worked out the Math (Matrix representation). I have vectorised the back propagation mechanism!

Now you might wonder from where did I arrive at my Milestone matrix notation. If you expand on that WThidden_output you will find out that this weights matrix is just a Transpose of the original weights Matrix during the forward propagation phase. This is a big step in understanding the way we distribute the errors into the subsequent hidden layers of the Neural Networks.

The hidden_output in my sheet above in the simple 2 layer, 2 node network mean that my hidden layer is also the input layer. The reason why I decided to call it as hidden_output is for the fact that a typical Neural Network will not just have an input and an output layer, but also several hidden layers. So don't be confused with the hidden_output, well I could have as well called it input_output! But nevertheless, you get the message! Don't you?

Understanding Feed Forward Neural Network Architectures


I have been reading through the architectures of Neural Networks and wanted to grasp the idea behind calculating the weights in a Neural Network and as you can see in the image below is a simple 2 node 2 layer Neural Network.

As you can see that for simplicity sake, I have just used a 2 node 2 layer network, but the same holds true for any sized Neural Network. It's just that the size of the Matrix increases proportionally to the number of inputs! The concepts outlined in the image holds gold!

Math behind simple Linear Regression


I have been wondering on how the math behind a Linear regression works as in most of the ML books that you encounter, the focus will be on giving you a linear equation and just plugging this equation in a Python library to solve for the slope and bias and then use it to predict the new values. It is very rare that they show you how to find the m and b values. So here in a piece of paper, I decided to try that out and it worked out very well! So if you want to learn it, try to understand what partial derivatives are!

In the above solution, I have just solved for m, which is the slope term in a Linear Regression. You can apply the same technique to solve for b! So what you effectively do is to differentiate one term while treating the others as a constant. In simple terms, this is called a partial derivative. A derivative is a measure of something that changes while a partial derivative is a measure of something that changes while treating everything else in this world as a constant! It's that simple! More on Partial Derivatives

What I have shown you here is a Simple Linear Regression, but the technique applies equally as good as a multi variate Linear Regression! Math is fucking fun - Once you understand it!

My Trek Remedy clocks 1000 km this season


So as of today, I managed to clock on record a total of 1000 Km with my Trek Remedy 8. It is obviously much higher than that as for several rides I haven't turned my Sigma odometer device. But for the record, the mileage is 1000 Km for this year so far.

I have explored much of the singles trails around my area! It was fun riding this bike! I hope to do even more miles for the next season!

Yet another impact on the injured knee! Why would fate have it this way?


Last Thursday, I took a day off. As the weather was good I decided to take my kids and my mom to Apeldoorn in the Netherlands. We wanted to visit the theme park Julianatoren and spend the day there. My mom is here with us in Germany to visit us and to take care of the kids during their summer vacation. Apeldoorn is a 2 hour drive from where we live. As usual, we did not manage to start early and ended up reaching our destination around 12:00. By this time, the park was already crowded. Thinking that my mom could not walk for extended periods in the park, I reserved a battery operated wheel chain the day before - only to realise later that it was a big mistake!

I reached the entrance, paid the entry, went and took possession of the battery operated wheelchair. To me it was simple to operate as explained by the service personnel at the park. My mom nodded her head as though she understood everything! She sat on it, drove for about 50 meters after which my elder daughter wanted to use the rest room. So I ordered my mom to park the wheelchair in a corner so that it does not hinder the pedestrians and other visitors to the park. She abided by what I said and started to come towards one corner. I was standing with my younger daughter in my arms and while my mom was navigating the wheelchair to the corner, my elder daughter was trying to help her giving directions on how to steer and corner. This somehow confused my mom and she gave full throttle and guess what, she whacked on my leg. As I was standing sideways with my left side of my body facing the oncoming wheelchair, my mom was dashing against my left leg and I started to scream with pain and told her to stop which panicked her even further and she instead of taking her hands off the accelerator, she accelerated even further and gave couple more strikes with the wheelchair - the impact being taken by my already injured left leg. Since I do not have my ACL, my left leg would easily bend against this impact and this caused immense pain. Luckily for me the wheelchair came to a stop by dashing against an elevated floor, if not I would have been probably fatally injured! All this while my younger daughter was in my arms. All this happened within a couple of seconds!

The impact of this freaky accident meant that I had to use my crutches for the next couple of days and also use a knee bandage like this one here to stabilise my knee. Subsequently the pain kind of went away after a couple of days, but I missed out on those bike rides that I have originally planned! I will henceforth be extra cautious when out in such situations.

So after all this drama at the park, it was me who was using the wheelchair at the park and my mom was walking comfortably! I shouldn't have hired the wheelchair in the first place and I shouldn't have given it to my mom to ride it without her having enough practice! Lessons learned!

Send it... yep! Sendin it...


What does "Send it mean? It's a jargon used within the Mountain Biking community, especially the Downhill community that indicates that you are about to hit a jump and that you have to clear the jump! Jumpin on the trails requires technique and a lot of practice. One way to practice the technique is to do it using some artificially made ramps! That exactly was my DIY project this weekend! I went to the local store "Bauhaus" as it is called here in Germany and shopped for some wood and nails.

The dimensions for the ramp I wanted are {h = at least 30cm, l = at least 60cm}

I made the choice that my ramp should be at least 30cm from ground so that I have some decent enough lift off! For this to happen, I had to buy a single piece of wood and have it cut into pieces! I would then assemble the pieces together using the nails!

Here are some pictures!
It was fun to build it and it roughly took me 40 minutes to get it done! It is not perfect, but for what it costed me (roughly 15 Euros), this is absolutely fine! I will probably test it the next days and I'm sure I'll have fun with it!

What it feels like getting a black out


Today, I drove to Düsseldorf to ride my MTB on the Aaper wald. I made a nasty mistake of not having enough breakfast. It started good, but after reaching the foot of the hill, I started to ride the up-hill section. After pedalling for about a minute or two, I started to notice that my legs were pounding hard. I feel that this must be the activity of my heart trying to pump more blood to my legs to feed them with power. After a while, the slope got even steeper and I could not ride it anymore, so I got down and started to tow my bike. It was demanding! Once I reached the peak, I started to get tired and felt a pounding sensation on my face and I started to feel dizzy. It was within seconds, I blacked out!

The following event is as told by my friend who was with me for the ride!

I fell down un-conscious, my bike fell over me. My fried then took the bike away and called me twice after which I got my consciousness back and got up! It was roughly less than 10 seconds where I went un-conscious! My friend told me that while I was un-conscious, my eyes were still open and he was not sure if I was breathing or not! I'm glad that I came back and I'm thankful to my friend that he called my name which I feel was the trigger to get me back to consciousness! After waking up, I felt as though I was sleeping for longer and while I was un-conscious, I was dreaming of something! Not sure what I was dreaming about. For about 20 to 30 seconds of waking up, I was still a bit dizzy and I was slowly getting back to normal. I had some water to drink and was just asking my friend about what happened to me.

It was after this incident, we drove through the forest for about 16km, both uphill and downhill where I was shredding the descents!

Back with MTB Tours and Downhill rides


Mountain biking is addictive! Worse than drugs! After a couple of months of staying away from my MTB, I'm back with it and I'm looking forward to this season to ride as much as possible and more importantly, without any accidents and injuries! I got my bike serviced and about a week ago. This is how "The Beast" looked after returning back from the service!

I took it for a spin in the trails around Düsseldorf. I drove for about 30+ km and occasionally I had some strange feelings on my injured knee. The weather just got better that day. It was drizzling and the days before this, we had the storm "Eberhardt" that wrecked havoc near the area! As a result of all this, the trails were mushy and slippery.

At one point, when riding uphill, I got stuck between the roots and I could not pedal anymore as it was too steep. While I tried to put my left leg down to balance, I felt the buckling sensation! This is when I realised that I've got to be extra careful when riding technical trails and probably I will think of using a Knee protector / brace (like the Don Joy brace) that I had during the initial days of my ACL injury! But nevertheless, I continued and was able to complete the planned route. Overall I had a wonderful ride that day and I'm eagerly looking forward to discovering new trails and make new friends!

I would like to share an interesting app that I use to plan my MTB tours - Komoot is a social cycling app that you can use to plan your tours. Once you do so, you have the option to export your tour as a gpx data and import this into any navigation device. This would then give you a turn-by-turn navigation on your bike. I find this really helpful as I can now ride with confidence and just take any unknown terrain!

4 Months Update of my Knee after an ACL injury


It is now roughy 4 months since I injured my knee and the recovery so far has been very positive. On the downside, I have the feeling that my knee is not like it was before! Occasionally I get a strange wobbly feeling on my injured knee!

So for the first 2 months, I was not able to fully stretch or bend my injured leg, but that got a lot better now. I can fully flex my left leg - as good as the right leg, but there is still a 10 to 20 degrees of deficit when bending it. I'm happy to have recovered to this extent. The last 2 weeks, I was back on my MTB - not yet on gnarly mountains, but on single trails. I was able to do a couple of rides stretching anywhere between 35 ad 40 km in a single sitting. I remember having a weird feeling on my left knee after the ride, but I believe it is just a one off thing that got eventually better after a couple of minutes!

I'm now terribly confused. The 2019 season would start in just under 2 months from now. Getting a surgery is not an option now as it would put me off by 6 months using my bike. What I'm extremely afraid of are those jumps and drops and what effect would they have on my knee. But nevertheless, I don't want to be left without driving this season. I will avoid doing stunts and just drive for the fun of it! Hope to have a fun filled MTB season for 2019!