Unsupervised Abstractive Opinion Summarization
by Generating Sentences with Tree-Structured Topic Guidance
Masaru Isonuma1
Junichiro Mori1,2 Danushka Bollegala3
Ichiro Sakata1
1The University of Tokyo, 日本
2 RIKEN, 日本
3 University of Liverpool, 英国
isonuma@ipr-ctr.t.u-tokyo.ac.jp
mori@mi.u-tokyo.ac.jp
danushka@liverpool.ac.uk
isakata@ipr-ctr.t.u-tokyo.ac.jp
抽象的
This paper presents a novel unsupervised
abstractive summarization method for opin-
ionated texts. While the basic variational
autoencoder-based models assume a unimodal
Gaussian prior for the latent code of sentences,
we alternate it with a recursive Gaussian mix-
真实, where each mixture component corre-
sponds to the latent code of a topic sentence
and is mixed by a tree-structured topic distribu-
的. By decoding each Gaussian component,
we generate sentences with tree-structured
topic guidance, where the root sentence con-
veys generic content, and the leaf sentences
describe specific topics. Experimental results
demonstrate that the generated topic sentences
are appropriate as a summary of opinionated
文本, which are more informative and cover
more input contents than those generated by
the recent unsupervised summarization model
(Braˇzinskas et al., 2020). 此外, 我们
demonstrate that the variance of latent Gauss-
ians represents the granularity of sentences, 一个-
alogous to Gaussian word embedding (Vilnis
and McCallum, 2015).
1
介绍
Summarizing opinionated texts, such as product
reviews and online posts on Web sites, has at-
tracted considerable attention recently along with
the development of e-commerce and social media.
Although extractive approaches are widely used
in document summarization (Erkan and Radev,
2004; Ganesan et al., 2010), they often fail to pro-
vide an overview of the documents, 特别
for opinionated texts (Carenini et al., 2013; Gerani
等人。, 2014). Abstractive summarization can over-
come this challenge by paraphrasing and general-
izing an entire document. Although supervised
approaches have seen significant success with the
development of neural architectures (See et al.,
2017; Fabbri et al., 2019), they are limited to
specific domains, 例如, news articles, where a large
945
number of gold summaries are available. 如何-
曾经, the domain of opinionated texts is diverse;
manually writing gold summaries is therefore
昂贵.
This lack in gold summaries has motivated prior
work to develop unsupervised abstractive summa-
rization of opinionated texts, 例如, 产品
reviews (Chu and Liu, 2019; Braˇzinskas et al.,
2020; Amplayo and Lapata, 2020). While they
generated consensus opinions by condensing in-
put reviews, two key components were absent:
topics and granularity (IE。, the level of detail). 为了
实例, 如图 1, a gold summary
of a restaurant review provides the overall impres-
sion and details about certain topics, such as food,
ambience, and service. 因此, a summary typi-
cally comprises diverse topics, some of which are
described in detail, whereas others are mentioned
concisely.
From this investigation, we capture the topic-
tree structure of reviews and generate topic sen-
时态, 那是, sentences summarizing specified
主题. In the topic-tree structure, the root sentence
conveys generic content, and the leaf sentences
mention specific topics. From the generated topic
句子, we extract sentences with appropriate
topics and levels of granularity as a summary. 关于-
garding extractive summarization, capturing top-
集成电路 (Titov and McDonald, 2008; Isonuma et al.,
2017; Angelidis and Lapata, 2018) and topic-tree
结构 (Celikyilmaz and Hakkani-Tur, 2010,
2011) is useful for detecting salient sentences. 到
the best of our knowledge, this is the first study
to use the topic-tree structure in unsupervised ab-
stractive summarization.
The difficulty of generating sentences with tree-
structured topic guidance lies in controlling the
granularity of topic sentences. Wang et al. (2019)
generated a sentence with designated topic guid-
安斯, assuming that the latent code of an input
sentence can be represented by a Gaussian mixture
计算语言学协会会刊, 卷. 9, PP. 945–961, 2021. https://doi.org/10.1162/tacl 00406
动作编辑器: Asli Celikyilmaz. 提交批次: 3/2021; 修改批次: 4/2021; 已发表 9/2021.
C(西德:2) 2021 计算语言学协会. 根据 CC-BY 分发 4.0 执照.
我
D
哦
w
n
哦
A
d
e
d
F
r
哦
米
H
t
t
p
:
/
/
d
我
r
e
C
t
.
米
我
t
.
e
d
你
/
t
A
C
我
/
我
A
r
t
我
C
e
–
p
d
F
/
d
哦
我
/
.
1
0
1
1
6
2
/
t
我
A
C
_
A
_
0
0
4
0
6
1
9
6
2
4
6
4
/
/
t
我
A
C
_
A
_
0
0
4
0
6
p
d
.
F
乙
y
G
你
e
s
t
t
哦
n
0
7
S
e
p
e
米
乙
e
r
2
0
2
3
数字 1: Outline of our approach. (1) The latent distribution of review sentences is represented as a recursive
GMM and trained in an autoencoding manner. 然后, (2) the topic sentences are inferred by decoding each
Gaussian component. An example of a restaurant review and its corresponding gold summary are displayed.
模型 (GMM), where each Gaussian component
corresponds to the latent code of a topic sentence.
While they successfully generated a sentence relat-
ing to a designated topic by decoding each mixture
成分, modelling the sentence granularity in
a latent space to generate topic sentences with mul-
tiple granularities remains to be realized.
To overcome this challenge, we model the sen-
tence granularity by the variance size of the latent
代码. We assume that general sentences have more
uncertainty and are generated from a latent distri-
bution with a larger variance, analogous to Gauss-
ian word embedding (Vilnis and McCallum, 2015).
Based on this assumption, we represent the latent
code of topic sentences with Gaussian distribu-
系统蒸发散, where the parent Gaussian receives a larger
variance and represents a more generic topic sen-
tence than its children, 如图 1. 到
obtain the latent code characterized above, 我们
introduce a recursive Gaussian mixture prior to
modeling the latent code of input sentences in
reviews. A recursive GMM consists of Gaussian
components that correspond to the nodes of the
topic-tree, and the child priors are set to the in-
ferred parent posterior. Because of this configu-
配给, the Gaussian distribution of higher topics
receives a larger variance and conveys more gen-
eral content than lower topics.
The contributions of our work are as follows:
• Experiments demonstrate that the generated
summaries are more informative and cover
more input content than the recent unsu-
pervised summarization (Braˇzinskas et al.,
2020).
2 Preliminaries
Bowman et al. (2016) adapted the variational
autoencoder (VAE; Kingma and Welling, 2014;
Rezende et al., 2014) to obtain the density-based
latent code of sentences. They assume the gener-
ative process of documents to be as follows:
For each document index d ∈ {1, . . . , D}:
For each sentence index s ∈ {1, . . . , Sd} in d:
1. Draw a latent code of the sentence xs ∈ Rn:
xs ∼ p(xs)
(1)
2. Draw a sentence ws:
ws|xs ∼ p(ws|xs) = RNN(xs)
(2)
(西德:2)
|w
