In terms of transition matrices what are actually Markov chains?
What are Markov chains and how are they used?
-warning: the following is the ramblings of a stats nerd. Not all is requisite knowledge for methods 3/4, however is useful to think about for Markov chains-
Before we discuss what a Markov chain is, it is useful to discuss what the
Markov property is. Any event can be found dependent on an event that happened earlier - a simple example is how well you do in your methods SAC dependent on the amount of time you studied for it. HOWEVER, if an event is found to be dependent on its present state (i.e, what's happening now), and nothing else, we say that it has the
Markov property. An example of this is how well your sporting team plays in a game, based on the amount of times they practiced this week. Note that studying may not exhibit the Markov property, as you may have studied things last year and not need to study them this time.
A
Markov chain is an abuse of the
Markov property. Let's say we have a football team, and if they practice three times a week, they have a 0.8 probability of winning a match. However, if they practice less than 3 times a week, the probability of winning is only 0.3. We can display this system as follows:
=0.8\:\:\: \mathbb{P}(W|T^c)=0.3<br />\\ \mathbb{P}(L|T)=0.2\:\:\: \mathbb{P}(L|T^c)=0.7)
Looks an awful a set of simultaneous equations, eh? Well, using the law of total probability, we get:
=\mathbb{P}(W|T)\mathbb{P}(T)+\mathbb{P}(W|T^c)\mathbb{P}(T^c)<br />\\ \mathbb{P}(L)=\mathbb{P}(L|T)\mathbb{P}(T)+\mathbb{P}(W|T^c)\mathbb{P}(T^c))
This is definitely a set of simultaneous equations - so, let's put it in matrix form:
\\ \mathbb{P}(L)\end{bmatrix}=\begin{bmatrix}\mathbb{P}(W|T)&\mathbb{P}(W|T^c)\\\mathbb{P}(L|T)&\mathbb{P}(L|T^c)\end{bmatrix}\begin{bmatrix}\mathbb{P}(T)\\\mathbb{P}(T^c)\end{bmatrix})
Now, what if we had an event that when measured, told us about the future? For example, above we could figure out if the team won based on how much they trained - but what if we could figure it out based on if they won the preceding match? Seems somewhat unrealistic, but from a psychological view, it could definitely happen. In fact, my super team of psychologists got together, and they reckon that if a team won a game the previous week, they were 75% more likely to win a game in the future. If they lost, they were 60% more likely to fail the next time.
Using this information, all of a sudden our matrix equation becomes:
\\ \mathbb{P}(L_2)\end{bmatrix}=\begin{bmatrix}\mathbb{P}(W_2|W_1)&\mathbb{P}(W_2|L_1)\\\mathbb{P}(L_2|W_1)&\mathbb{P}(L_2|L_1)\end{bmatrix}\begin{bmatrix}\mathbb{P}(W_1)\\\mathbb{P}(L_1)\end{bmatrix})
But remember, this follows the
Markov property. So, we should be able to generalise this for any
ith game. Indeed, we can, and it's very similar to what is depicted above:
\\ \mathbb{P}(L_{i+1})\end{bmatrix}=\begin{bmatrix}\mathbb{P}(W_{i+1}|W_i)&\mathbb{P}(W_{i+1}|L_i)\\\mathbb{P}(L_{i+1}|W_i)&\mathbb{P}(L_{i+1}|L_i)\end{bmatrix}\begin{bmatrix}\mathbb{P}(W_i)\\\mathbb{P}(L_i)\end{bmatrix})
THIS is a
Markov chain. It's an abuse of the Markov property to find the probability of an outcome at any i-th point in time. The Markov chain has three components:
"i+1"th state = transition matrix * "i"th state
Where the transition matrix is composed as above. Note: the Markov chain above is called a 2-state Markov chain, these are the only kind you need to be concerned about in methods.
Any more questions, or want some further explanations, just ask~
Home
Help
Search
Login
