saguar1

YANAI Lab.

電気通信大学 総合情報学科/大学院 総合情報学専攻 メディア情報学コース 柳井研究室
電気通信大学 > 情報工学科 > コンピュータ学講座 > 柳井研究室 > 研究紹介  

$B%i%C%7%e1GA|$N<+F0MWLs(B

$BLn8}(B $B82;L(B

2008$BG/(B 2$B7n(B 7$BF|(B




1 $B$O$8$a$K(B

$BK\8&5f$G$O!$9q:]1GA|=hM}%o!<%/%7%g%C%W(BTRECVID$B$G(B2007$BG/$+$i;O$^$C$?(B $B1GA|<+F0MWLs%?%9%/!J(Brushes summarization$B!K$K$D$$$F1]$B$r;29M$K%7%g%C%HJ,3d$7$?1GA|$r?'!$F0$-!$4iFCD'$K4p$E$$$F%/%i%9%?%j%s%0$9$k$3$H(B $B$K$h$C$F!$1GA|MWLs$r

2 TRECVID$B$K$D$$$F(B

TRECVID$B$H$O1GA|%3!<%Q%9$rMQ$$$?>pJs8!:w$N$?$a$N6%Ah7?%o!<%/%7%g%C%W$GJF(B $B9q$N(BNIST(National Institute of Standards and Technology)$B$Ne$G$"$k!%(B

TRECVID 2007$B$K$*$$$F0J2<$N(B4$B$D$N%?%9%/$,@_Dj$5$l$?!%(B

  • Shot boundary detection($B%7%g%C%H6-3&8!=P(B)
  • High-level feature extraction($B9b
  • Search($B8!:w(B)
  • Rushes summarization($B%i%C%7%e1GA|MWLs(B)
$BK\8&5f$G:#2s Rushes summarization$B$OM?$($i$l$?%i%C%7%e1GA|(B(MPEG-1)$B$r7h$a$i$l$?;~4V0J2<(B(2007$B$K$*$$$F$O(B4%$B0J2<(B)$B$K<+F0$GMW(B $BLs$9$k%?%9%/$G$"$k!%%i%C%7%e1GA|$H$OL$JT=8$N1GA|$N$3$H$G$"$j!$GPM%$N(BNG$B%7!<(B $B%s$J$I$N7+$jJV$7%7!<%s!$%+%a%i$,8GDj$5$l$F$$$FD9$$4VF0$-$,$J$$%7!<%s$r(B $B4^$s$G$$$k1GA|$N$3$H$G$"$k!%(B

$B$3$N%?%9%/$K$*$1$kI>2AJ}K!$O!$%F%-%9%H7A<0$N(Bground truth$B$H$N0lCWN($d!$%j%C%+!<(B $B%H<\EY$K$h$kMWLs$H$7$F$N8+0W$5$dL5BL$N>/$J$5$N$h$&$J2]$B!%(B

$B?^(B1$B!$(B2$B$O$=$l$>$l%i%C(B $B%7%e1GA|$N(B10$BIC$4$H$N%U%l!<%`$H

$B?^(B 1: $B%i%C%7%e1GA|$N(B10$BICKh$N%U%l!<%`(B
\includegraphics[width=0.6\textwidth]{rush.eps}

$B?^(B 2: $BMWLs1GA|$N(B10$BICKh$N%U%l!<%`(B
\includegraphics[width=0.45\textwidth]{sum.eps}



$BI=(B 1: Ground truth$B$NNc(B
Shot of trees
Woman towards camera, stops and talks
Woman turns around and walks down footpath

3 $B%"%k%4%j%:%`(B

$B$3$3$G$OK\%7%9%F%`$N%"%k%4%j%:%`$N35MW$K$D$$$F@bL@$9$k!%?^(B3$B$O%7%9%F%`$N35(B $BMW$rI=$7$F$$$k!%(B

$B:G=i$KM?$($i$l$?%S%G%*$r?'FCD'$r$b$H$KA08e$N%U%l!<%`$rHf3S$7!$?'%R%9%H(B $B%0%i%`$N:9J,$,ogCM0J2<$J$i%7%g%C(B $B%HJ,3d$9$k!%$^$?3F%7%g%C%H$N?'FCD'$+$i%/%i%9%?%j%s%0$r9T$$!$$=$l$>$l$N%/%i(B $B%9$+$i:G$bD9$$%7%g%C%H$rBeI=$H$7$FA*$s$G$$$/!%(B

$B$=$N:](B,$B%V%i%C%/%U%l!<%`$d!$%+%i!<%P!<$J$I$N%8%c%s%/%7%g%C%H$N8!=P$r%/(B $B%i%9C10L$G?'FCD'$rMQ$$$F9T$&!%(B

$B$3$N$h$&$K$7$FF@$i$l$?3F%/%i%9%?$NBeI=$+$i:#EY$O?'FCD'(B,$B4iFCD'(B,$BF0$->pJs$r(B $BCj=P$7$J$,$i!$3F%/(B $B%i%9%?$NBeI=$r$=$l$>$l0lICC10L$KJ,3d$9$k!%$?$@$7$3$N:]$K(BLucas-Kanade$BK!(B[3]$B$G(B $B%*%W%F%#%+%k%U%m!<$r7W;;$7$F!$$"$k0lDj0J>e$NF0$-$,$"$C$?>l9g$K$=$l$O0lO"(B $B$NF0:n$NESCf$G$"$k$H9M$(!$F0$-$,0lDj0J2<$K$J$k$^$GJ,3d$r9T$o$J$$$h$&$K$9(B $B$k!%(B

$B$3$l$K$h$j(BCMU$B$N1]$B$K$*$$$F$O%+%a%i%b!<%7%g%s$N$_$N6/D4$G$"$C$?$,!$K\8&5f$G$O(B $BF0:n$b6/D4$G$-$k$h$&$K$9$k!%(B

$B$=$N0lICKh$KJ,$1$i$l$?%S%G%*$KBP$7$F?'FCD'$r85$K%*%j%8%J%k%S%G%*$N(B4%$B0J(B $B2<$K$J$k$h$&$K(Bk$B$NCM$r@_Dj$7$F!$(Bk-means$B%"%k%4%j%:%`$G%/%i%9%?%j%s%0$7$F$$(B $B$/!%(B

$B3F%/%i%9%?$NBeI=$O!$$G$-$k$@$1F0$-$,$"$k$b$N!$?M$,1G$C$F$$$k$b$N$,$[$7$$$N(B $B$G!$F0$->pJs$H4i>pJs$rMQ$$$F%/%i%9%?$NBeI=$r7hDj$9$k!%3F%/%i%9%?$NBeI=$r(B $B;~4V=g$K$D$J$.$"$o$;$F!$MWLs1GA|$H$9$k!%(B

$B?^(B 3: $B%"%k%4%j%:%`35MW(B
\includegraphics[width=0.69\textwidth]{flow.eps}

4 $BFCD'NL(B

$BK\>O$G$O%7%g%C%H$rI=$9FCD'NL$N;;=PJ}K!$K$D$$$F@bL@$9$k!%(B

4.1 $B0LCV>pJsIU%+%i!<%R%9%H%0%i%`(B

$B?'FCD'$H$7$F$O(B$3 \times 3$$B$KJ,3d$7$?2hA|$N(BRGB$B%+%i!<%R%9%H%0%i%`$r;HMQ$9$k!%(B $B3F%7%g%C%H$N?'FCD'(BC$B$O<0(B(1)$B$GDj5A$9$k!%(B

$\displaystyle C =\frac{1}{F} \sum^F_{i=2}\sum^3_{x=1}\sum^3_{y=1}\sum^{64}_{k=0}
H_{i_{{xy}_k}} +H_{i-1_{{xy}_k}}$     (1)


$B$?$@$7(BF$B$O%7%g%C%HCf$K4^$^$l$k%U%l!<%`$N?t$r!$(B $H_{i_{{xy}_k}}$$B$O%7%g%C%HCf$N(Bi $BHVL\$N%U%l!<%`$N3J;R(B(x,y)$B$N%R%9%H%0%i%`$N(Bk$BHVL\$NMWAG$G$"$k$3$H$r<($7$F$$$k!%(B

4.2 $BF0$->pJs(B

Lucas-Kanade$BK!(B[3]$B$K$h$C$FO"B3$9$k%U%l!<%`4V$N%*%W%F%#%+%k%U%m!<$r7W;;$9$k!%(B 2$B$D$N%U%l!<%`4V$K$*$1$kF0$->pJs(B$M_{i}$$B$O<0(B(2)$B$GDj5A$9$k!%(B

$\displaystyle M_{i}= \frac{1}{N}\sum^{N}_{k=1}(x_{k,i}-x_{k,i+1})^2+(y_{k,i}-y_{k,i+1})^2$     (2)


$B$?$@$7(BN$B$O8+$D$+$C$?A4$F$N%*%W%F%#%+%k%U%m!<$N8D?t$r!$(Bx$B!$(By$B$O$=$l$>$l$N:B(B $BI8$rI=$7$F$$$k!%Nc$($P(B$x_{k,i}$$B$O!$(Bi$BHVL\$N%U%l!<%`$N(Bk$B8D$a$N%U%m!<$N(Bx$B:BI8(B $B$rI=$9$b$N$G$"$k!%(B

$B$=$7$F%7%g%C%H$H$7$F$NF0$->pJs!$(BALL_M$B$O<0(B(3)$B$GDj5A$5$l$k!%(B

$\displaystyle ALL\_M= \frac{1}{F-1}\sum^{F}_{k=1}M_{k}$     (3)


$B$?$@$7(BF$B$O%S%G%*$K4^$^$l$kA4$F$N%U%l!<%`$N?t$G$"$k!%(B

4.3 $B4iFCD'(B

$B4i$NG'<1$O(BOpenCV[4]$B$N4i2hA|8!=P%W%m%0%i%`%k!<%A%s$rMxMQ$9$k!%$^$?4i$,8!=P$5(B $B$l$?$b$N$K$O!$=E$_(BW($Bl9g(BW=1$B$G$"(B $B$k!%(Bk-means$B$G%/%i%9%?%j%s%0$7$?$"$H$K3F%/%i%9%?$NBeI=$r
$\displaystyle RE = ALL\_M \times W$     (4)


5 $B

$B
  1. $BFCD'NL$,?'$N$_(B
  2. $BFCD'NL$,?'(B+$BF0$-(B
  3. $BFCD'NL$,?'(B+$BF0$-(B+$B4i(B


$B 5.1 $B $B.(B11$BJ,!$J?6QCM(B $B$OLs(B21$BJ,$G$"$C$?(B.

5.2 $BI>2A4p=`(B

$BK\8&5f$G$ODs0F%7%9%F%`$NI>2A4p=`$H$7$F(BTRECVID$B$NI>2AJ}K!$NCf$+$i(B3$B$D$NI>2A4p=`$rMQ$$(B $B$?!%0l$D$a$O$=$N%S%G%*$KBP$9$k(Bground truth$B$,$I$l$@$1$N3d9g$G4^$^$l$F$$$k$+(B $B$rI=$9(BIN$BCM!$(B2$B$D$a$O%*%j%8%J%k%S%G%*$KBP$7$F2?%Q!<%;(B $B%s%H$NMWLs$K$J$C$F$$$k$+$r<($9(BDU$BCM!$(B3$B$D$a$,%7%9%F%`$N

5.3 $B

$B7k2L$OI=(B2$B!$(B3$B!$(B4$B!$$G<($9$h$&$K$J$C$?!%$?$@$7MQ$$$?%^%7%s$O(BPentium D 3.2GHz$B$G$"$k!%(B


$BI=(B 2: $B?'>pJs(B
  $B;~4V(B[s] IN DU[%] SYS[s]
rush01 2189 0.49 3.9 1488
rush02 2037 0.53 3.8 1386
rush03 721 0.61 3.7 613
rush04 738 0.38 10.4 1347
rush05 1951 0.63 3.8 1327
rush06 693 0.46 10.8 1348
rush07 743 0.62 3.7 525
rush08 767 0.42 9.5 1219
rush09 1702 0.66 3.8 1014
$BJ?6Q(B 1282 0.50 5.0 1141


$BI=(B 3: $B?'>pJs(B+$BF0$->pJs(B
  $B;~4V(B[s] IN DU[%] SYS[s]
rush01 2189 0.53 3.5 1999
rush02 2037 0.62 3.6 1608
rush03 721 0.46 3.7 864
rush04 738 0.50 7.7 1549
rush05 1951 0.60 3.5 1590
rush06 693 0.84 10.8 1719
rush07 743 0.75 3.7 657
rush08 767 0.36 5.4 1316
rush09 1702 0.66 3.6 1118
$BJ?6Q(B 1282 0.55 4.4 1338


$BI=(B 4: $B?'>pJs(B+$BF0$->pJs(B+$B4i>pJs(B
  $B;~4V(B[s] IN DU[%] SYS[s]
rush01 2189 0.62 3.6 2051
rush02 2037 0.60 3.3 1766
rush03 721 0.52 3.7 856
rush04 738 0.56 7.8 1540
rush05 1951 0.63 3.5 1590
rush06 693 0.76 10.8 1735
rush07 743 0.75 3.7 663
rush08 767 0.42 5.2 1298
rush09 1702 0.75 3.8 1308
$BJ?6Q(B 1282 0.60 4.3 1357


3$B$D$N%7%9%F%`$H$b(Brush08$B$N(BIN$BCM$,Hf3SE*Dc$$!%$3$l$O$3$N%S%G%*$,A4BN$rDL$7$F9u$$?'$,B?$+$C$?$?$a$G$"$k!%%7%9%F%`$N$J$+$G:G$b=EMW$G$"$k%/%i%9%?%j%s%0$O?'>pJs$N$_$G9T$o$l$F$$$k$N$G?'$NJQ2=$,>/$J$$%S%G%*$N(BIN$B$N@:EY$ODc$/$J$k$H9M$($i$l$k!%$^$?(B3$B$D$N%7%9%F%`$H$b(BDU$BCM$O(Brush04$B!$(Brush06$B!$(Brush08$B$K$*$$$F5,Dj$N;~4V$rBgI}$K1[$($F$7$^$C$F$$$k!$$=$N$H$-$N(BIN$BCM$r8+$F$$$C$?>l9g$bJ?6Q$KHf$YDc$/$J$k798~$,$"$k!%(B

$B%7%9%F%`Kh$K8+$F$$$/$H!$(BIN$BCM$OF0$-FCD'$r2C$($?:]$K(B5%$B!$$5$i$K4iFCD'$r2C$($?:]$K(B5%$B>e$,$C$F$$$k!%$3$N$3$H$+$i!$$3$N%?%9%/$G$OF0$-$H4iFCD'$r $B$^$?K\8&5f$,;29M$K$7$?(BCMU$B$N1]$B$N(BIN$BCM$O(B0.59$B$HK\%7%9%F%`$H$[$\F1$8CM$H$J$C$?!%$3$l$OK\8&5f$G%/%i%Q!<%\!<%I8!=P$r9T$C$F$$$J$$$J$I(B $B!$%8%c%s%/%7%g%C%H8!=P$,IT40A4$G$"$k$3$H$K5/0x$7$F$$$k!%(B

$Bl9g!$:G9b$N%A!<%`$,(B0.70$B!$Cf4V$N%A!<%`$,(B0.47$B$G$"$C$?$N$G!$K\%7%9%F%`$N(B0.60$B$ONI$$7k2L$K$J$C$?$H$$$($k!%$7$+$7(BDU$BCM$G$O$I$N;22C

6 $B9M;!(B

$BpJs$rCj=P(B $B$9$k:n6H$,%7%g%C%H8!=P$H(Bk-means$B$NFCD'$H$G=EJ#$7$F$$$k$3$H$G$"$k!%(B

$BFs$DL\$H$7$F$O!$MWLs$H$7$F8+Fq$/$J$C$F$$$k!%0lICKh$K>lLL$,@Z$jBX(B $B$o$C$F$7$^$&$N$G!$8+$F$$$kB&$bM}2r$9$k$3$H$,BgJQ$K$J$C$F$$$k!%(B

$B$^$?(BIN$BCM$K$*$$$F!$(Bground truth$B$NFbMF$,(B''Shot of tree''$B$N$h$&$J$b$N$O@:EY$,(B $BHf3SE*9b$+$C$?$,!$(B''Woman exit left''$B$N$h$&$JFbMF$N$H$-!$:8$K9T$/ESCf$GlLL$K@Z$jBX$o$C$F$7$^$&ItJ,$,B?$+$C$?!%$3$l$OF0$->pJs$,IT40A4$G$"$k$3(B $B$H$r<($7$F$$$k!%$3$N$3$H$OF0$->pJs$rlLL$,B?$/8+$i$l$?!%(B

$B99$K%/%i%9J,$1$O?'$rCf?4$K9T$C$F$$$k$N$G!$(Brush08$B$N$h$&$K0E$$>lLL$,BgH>$r(B $B$7$a$F$$$kF02h$K4X$7$F!$@:EY$,2<$C$F$7$^$C$F$$$k!%(B

7 $B$*$o$j$K(B

$BK\8&5f$G$O(BTRECVID$B$N%i%C%7%e1GA|MWLs%?%9%/$K$D$$$FpJs$r$b$H$K%/%i%9%?%j%s%0$5$l$?%7%g%C%H$r!$F0$-!$4i>pJs$rMQ$$$F=E$_$E$1$7!$$=$N=E$_$+$iBeI=%7%g%C%H$rA*$S;~4V=g$K$D$J$.$"$o$;$k $B$=$N7k2L$H$7$F!$F0$-!$4i>pJs$rMQ$$$k$3$H$O%7%9%F%`$N@:EY$r8~>e$5$;$k$3$H$,J,$+$C$?!%$^$?(BIN$BCM$K4X$7$F$O(BTRECVID$B;22C $B:#8e$N2]Bj$H$7$F!$%/%i%C%Q!<%\!<%I$N$h$&$J%8%c%s%/%7%g%C%H$N8!=P$rpJs$N2~NI$J$I$N!$FCD'NL$N2~NI!$2;>pJs$J$I$N8=:_L$;HMQ(B $B$NFCD'NL$NDI2C$,5s$2$i$l$k!%(B

$BJ88%L\O?(B

1
A. Hauptmann, M. Christel, W. Lin, B. Maher, J. Yang, R. Baron, and G. Xiang.
Clever clustering vs. simple speed-up for summarizing rushes.
In Proc. of the international workshop on TRECVID video summarization, pp. 20-24, 2007.

2
P. Over, A. Smeaton, and P. Kelly.
The trecvid 2007 bbc rushes summarization evaluation pilot.
In Proc. of the international workshop on TRECVID video summarization, pp. 1-15, 2007.

3
A. Hauptmann, M. Christel, W. Lin, B. Maher, J. Yang, R. Baron, and G. Xiang.
Clever clustering vs. simple speed-up for summarizing rushes.
In Proc. of the international workshop on TRECVID video summarization, pp. 20-24, 2007.

4
D. Le and S. Satoh.
National institute of informatics, japan at trecvid 2007: Bbc rushes summarization.
In Proc. of the international workshop on TRECVID video summarization, pp. 70-73, 2007.