离散变量 | 连续变量 |
---|---|
H ( X ) = ∑ x i ∈ X P ( x i ) log 1 P ( x i ) H(X)=sum_{x_{i} in X} P(x_{i}) log frac{1}{P(x_{i})} H(X)=∑xi∈XP(xi)logP(xi)1 | H ( X ) = ∫ P ( x ) ⋅ log 1 P ( x ) d x H(X)=int P(x) cdot log frac{1}{P(x)} d x H(X)=∫P(x)⋅logP(x)1dx |
H ( X ∣ Y ) = ∑ x i ∈ X ∑ y j ∈ Y P ( x i , y j ) log 1 P ( x j ∣ y i ) H(X mid Y)=sum_{x_{i} in X} sum_{y_{j} in Y} P(x_{i}, y_{j}) log frac{1}{P(x_{j} mid y_{i})} H(X∣Y)=∑xi∈X∑yj∈YP(xi,yj)logP(xj∣yi)1 | 略 |
H ( X , Y ) = ∑ x i ∈ X ∑ y j ∈ Y P ( x i , y j ) log 1 P ( x i , y j ) H(X, Y)=sum_{x_{i} in X} sum_{y_{j} in Y} P(x_{i}, y_{j}) log frac{1}{P(x_{i}, y_{j})} H(X,Y)=∑xi∈X∑yj∈YP(xi,yj)logP(xi,yj)1 | 略 |
I ( X ; Y ) = H ( X ) − H ( X ∣ Y ) = H ( Y ) − H ( Y ∣ X ) = H ( X ) + H ( Y ) − H ( X , Y ) I(X ; Y)=H(X)-H(X mid Y)=H(Y)-H(Y mid X)=H(X)+H(Y)-H(X, Y) I(X;Y)=H(X)−H(X∣Y)=H(Y)−H(Y∣X)=H(X)+H(Y)−H(X,Y)
例题1:
令
{
u
1
,
u
2
,
…
,
u
8
}
left{u_{1}, u_{2}, ldots, u_{8}right}
{u1,u2,…,u8} 为一等概消息集, 各消息相应被编成下述二元码字:
u
1
=
0000
,
u
2
=
0011
,
u
3
=
0101
,
u
4
=
0110
u_{1}=0000, quad u_{2}=0011, quad u_{3}=0101, quad u_{4}=0110
u1=0000,u2=0011,u3=0101,u4=0110,
u
5
=
1001
,
u
6
=
1010
,
u
7
=
1100
,
u
8
=
1111
u_{5}=1001, quad u_{6}=1010, quad u_{7}=1100, quad u_{8}=1111
u5=1001,u6=1010,u7=1100,u8=1111
通过转移概率为p的BSC传输,求:
(1)接收到的第一个数字 0 与
u
1
u_{1}
u1 之间的互信息量。
(2)接收到的前四个数字 0000 与
u
1
u_{1}
u1 之间的互信息量。
(1)
p
(
0
)
=
1
8
(
1
−
p
)
×
4
+
1
8
p
×
4
=
1
2
p(0)=frac{1}{8}(1-p) times 4+frac{1}{8} p times 4=frac{1}{2}
p(0)=81(1−p)×4+81p×4=21
I
(
u
1
;
0
)
=
log
p
(
0
∣
u
1
)
p
(
0
)
=
log
1
−
p
1
2
=
1
+
log
(
1
−
p
)
Ileft(u_{1} ; 0right)=log frac{pleft(0 mid u_{1}right)}{p(0)}=log frac{1-p}{frac{1}{2}}=1+log (1-p)
I(u1;0)=logp(0)p(0∣u1)=log211−p=1+log(1−p) bit
(2)
p
(
0000
)
=
1
8
[
(
1
−
p
)
4
+
6
(
1
−
p
)
2
p
2
+
p
4
]
p(0000)=frac{1}{8}left[(1-p)^{4}+6(1-p)^{2} p^{2}+p^{4}right]
p(0000)=81[(1−p)4+6(1−p)2p2+p4]
I
(
u
1
;
0000
)
=
log
8
(
1
−
p
)
4
(
1
−
p
)
4
+
6
(
1
−
p
)
2
p
2
+
p
4
Ileft(u_{1} ; 0000right)=log frac{8(1-p)^{4}}{(1-p)^{4}+6(1-p)^{2} p^{2}+p^{4}}
I(u1;0000)=log(1−p)4+6(1−p)2p2+p48(1−p)4 bit
最后
以上就是受伤背包最近收集整理的关于熵、条件熵、联合熵、互信息关系及例题的全部内容,更多相关熵、条件熵、联合熵、互信息关系及例题内容请搜索靠谱客的其他文章。
发表评论 取消回复