Why Great Ideas Are Often Overlooked

Rietzschel, Eric F.; Nijstad, Bernard A.; Stroebe, Wolfgang
Published in:
The Oxford Handbook of Group Creativity
DOI:
10.1093/oxfordhb/9780190648077.013.11
IMPORTANT NOTE: You are advised to consult the publisher’s version (publisher’s PDF) if you wish to cite from
it. Please check the document version below.
Document Version
Publisher’s PDF, also known as Version of record
Publication date:
2019
Link to publication in University of Groningen/UMCG research database
Citation for published version (APA):
Rietzschel, E. F., Nijstad, B. A., & Stroebe, W. (2019). Why Great Ideas Are Often Overlooked: A Review
and Theoretical Analysis of Research on Idea Evaluation and Selection. In P. B. Paulus, & B. A. Nijstad
(Eds.), The Oxford Handbook of Group Creativity (pp. 179–197). Oxford University Press.
https://doi.org/10.1093/oxfordhb/9780190648077.013.11
Copyright
Other than for strictly personal use, it is not permitted to download or to forward/distribute the text or part of it without the consent of the
author(s) and/or copyright holder(s), unless the work is under an open content license (like Creative Commons).
The publication may also be distributed here under the terms of Article 25fa of the Dutch Copyright Act, indicated by the “Taverne” license.
More information can be found on the University of Groningen website: https://www.rug.nl/library/open-access/self-archiving-pure/taverneamendment.
Take-down policy
If you believe that this document breaches copyright please contact us providing details, and we will remove access to the work immediately
and investigate your claim.
Downloaded from the University of Groningen/UMCG research database (Pure): http://www.rug.nl/research/portal. For technical reasons the
number of authors shown on this cover page is limited to 10 maximum.
Download date: 15-02-2025
The Oxford Handbook of Group Creativity and Innovation
Paul B. Paulus (ed.), Bernard A. Nijstad (ed.)
https://doi.org/10.1093/oxfordhb/9780190648077.001.0001
Published: 2019 Online ISBN: 9780190648091 Print ISBN: 9780190648077
CHAPTER
https://doi.org/10.1093/oxfordhb/9780190648077.013.11 Pages 179–197
Published: 09 May 2019
Abstract
Keywords: idea selection, evaluation, originality, feasibility, science, creativity, business, expertise,
regulatory focus
Subject: Organizational Psychology, Psychology
Series: Oxford Library of Psychology
Collection: Oxford Handbooks Online
11 Why Great Ideas Are Often Overlooked: A Review and
Theoretical Analysis of Research on Idea Evaluation and
Selection 
Eric F. Rietzschel, Bernard A. Nijstad, Wolfgang Stroebe
Group creativity is far more than the generation of ideas: Ultimately, creative ideas must be
recognized, valued, and selected. This chapter reviews and theoretically analyzes the relevant
literature on the recognition, evaluation, and selection of creative ideas in groups. In doing so, it
explains that both idea evaluation and idea selection show substantial room for improvement, with
idea selection being especially problematic. The chapter argues that the underlying problem in
eective idea evaluation and selection is the tension between originality and feasibility, and that
highly original ideas tend to be disliked or rejected because they are perceived to be risky and
unfeasible. Situational or personal factors that make implementation or feasibility concerns more
salient will therefore hinder creative idea selection. After discussing the literature, the chapter
addresses some possible solutions and directions for future research.
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Introduction
p. 179
Creativity is commonly dened as the production of ideas that are both novel (original, new) and useful
(appropriate, feasible) (e.g., Amabile, 1983; Sternberg & Lubart, 1999). Ideas that are original but not useful
are irrelevant, and ideas that are useful but not original are mundane (e.g., De Dreu, 2010). While this
denition is widely used in research, an important aspect of creativity is often ignored: Generating creative
ideas rarely is the nal goal. Rather, to successfully solve problems or innovate requires one or a few good
ideas that really work, and work better than previous approaches (see e.g., Rietzschel, Nijstad, & Stroebe,
2006). This requires that people evaluate the products of their own or each other’s imagination, and choose
those ideas that seem promising enough to develop further, and abandon those that are unlikely to be
successful. Thus, being creative does not stop with idea generation. In fact, the ability to generate creative
ideas is essentially useless if these ideas subsequently die a silent death.
Although the past decade has seen an increase in research on idea selection, it remains surprisingly
understudied. What is clear from the existing research, however, is that idea evaluation and selection are
highly challenging, and often form a real bottleneck for innovation. As we will show, creative ideas are
frequently not recognized as valuable, and often truly creative ideas are discarded (whereas conventional
ideas are selected). Therefore, generated creative potential often gets lost before implementation, and more
research is urgently needed to understand the conditions under which creative ideas are valued, selected,
and implemented.
Idea evaluation and selection can happen individually, but often takes place in group settings, such as
project teams, management teams, committees, or juries. Unfortunately, research on group-level
evaluation and selection of creative ideas is even scarcer than research at the individual level. In this
chapter, we therefore discuss relevant research at both the individual and the group level, and attempt to
extrapolate from individual-level work to group settings.
p. 180
We outline a basic model of idea selection that could serve as a starting point for future research, and use
this model to review and integrate the literature on idea evaluation and selection. The basic premises of our
model are that, rst, it is useful to distinguish between idea evaluation (recognition of creativity) on the one
hand, and idea selection (selection of one or several most creative ideas) on the other hand; second, both the
recognition and selection of creative ideas require the ability and motivation to do so, and both of these are
subject to individual dierences and situational constraints; third, the most important challenge in eective
recognition and selection of creative ideas is the tension between originality and feasibility: The more salient
feasibility or implementation concerns are, the less likely people will be to support or select highly original
ideas, because original ideas are inherently risky and carry many unknowns. We argue that this issue is
likely to be especially challenging for groups, but that groups also hold a unique potential to overcome this
tension by means of, for example, eective information exchange and redevelopment of ideas.
Before going more deeply into our model and summarizing the scientic literature, we give a brief overview
of historical cases, demonstrating that idea evaluation and selection are by no means trivial activities: Many
truly creative ideas were originally not recognized as creative, were rejected out of hand, or were simply
ignored.
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Identifying Creativity in Science
Identifying Creativity in the Arts
Creativity Is Not Enough: The Long Road from Creation to Success
To make contributions to any domain requires not only that creative ideas be generated, but also that these
ideas be recognized as creative, and adopted by relevant stakeholders (see, e.g., Csikszentmihalyi, 1999).
Unfortunately, suggesting a truly creative idea by no means guarantees that it will become successful.
Indeed, examples from a variety of domains suggest that the failure to recognize or select creative ideas is
often a bigger problem than generating or producing them in the rst place.
1
Arguably, one of the most important moments in scientic history has been the introduction of the theory
of evolution. This happened on July 1, 1858, when Darwin’s and Wallace’s evolutionary theories were (rst)
presented to the Linnean Society. However, in his presidential report on that year, given in May 1859,
president Bell of the Society wrote the following: “The year which has passed […] has not, indeed, been
marked by any of those striking discoveries which at once revolutionize, so to speak, the department of
science on which they bear” (see Guerrero, 2008).
Failure to recognize breakthroughs in science is not exceptional. For example, Mendel’s important and
seminal work on genetics was originally published in 1866, but was only cited about three times over the
next 35 years. One reason probably is that the title “Versuche über Panzenhybriden” (“Experiments on
plant hybridization”) did not immediately suggest a connection with heritability of traits.
Although these may seem extreme examples, more systematic evidence from the peer review process shows
that failure to recognize the quality of scientic work is a general problem. During the peer review process
reviewers evaluate a manuscript not only on methodological rigor but also on innovativeness: Does the
paper represent a novel contribution to the literature? Problematically, the agreement of journal reviewers
in their evaluation of scientic manuscripts has been found to be rather low, with typical correlations
around .20 (e.g., Petty, Fleming, & Fabrigar, 1999; Simonton, 2016).
The history of the arts is full of examples of creative products that were initially not recognized. Perhaps the
best-known example is Vincent Van Gogh, who only sold a single painting during his lifetime (The Red
Vineyard, sold in 1890 for 400 Francs). His paintings now are among the most expensive of all times.
Similarly, French Impressionists, such as Manet, Renoir, and Monet, had to ght an uphill battle to win
critical acclaim for their paintings or even to nd exhibition space (Farrell, 2001).
The same pattern can be seen in other domains of art, such as literature: Decision makers often failed to
identify creative products that ultimately were very successful. For example, J. K. Rowling received no less
than 12 rejections for Harry Potter and the Philosopher’s Stone, until the daughter of a Bloomsbury editor told
her father that she wanted to read the rest of the book. The editor nally agreed to publish the book, but
recommended to Rowling to take a day job, because she could not expect to earn a living with children’s
books. The Harry Potter series has (so far) sold over 450 million copies worldwide. Notable other examples
include James Joyce’s Ulysses (rejected so often that it was nally published in 1922 by a friend who
owned a bookshop), George Orwell’s classic Animal Farm (turned down as “unconvincing” by Faber & Faber
editor T. S. Eliot), Herman Melville’s Moby-Dick (which attracted the response “First, we must ask, does it
have to be a whale?”), Beatrix Potter’s The Tale of Peter Rabbit (which got rejected so often that Potter
ultimately decided to self-publish 250 copies—it has now sold 45 million). We can only guess how many
potential bestsellers were never published as a result of repeated rejections.
p. 181
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Identifying Creativity in Business
Evaluation and Selection: From Anecdote to Research
Cinematography and music show the same pattern. For example, when the rst Star Wars movie was
screened for studio audiences, there was no applause and Marcia Lucas (the wife of producer George Lucas)
broke out in tears (Kaminski, 2007). Lucas himself was sure that he had produced a op. A famous example
from the music industry is Decca Records’ refusal to sign up The Beatles, noting to their manager that
“guitar groups are on the way out, Mr. Epstein.” In classical music, Beethoven’s ninth symphony was
originally reviewed as boring and “frightful.” During the 1931 premiere performance of Igor Stravinsky’s
Rite of Spring (choreographed by Vaslav Nijinsky), one of the rst musical pieces to employ polytonality,
outraged audience members began to riot, attacking not only the performers but each other as well.
Of course, the diculty of identifying creative products is not limited to the arts and sciences. There are
numerous examples of the initial rejection of commercial products that nally turned out to be extremely
successful, as in the case of the Apple iPod and iPad. After the launch of the iPod, for example, responses
were mixed, with many analysts doubting whether Apple would be successful in entering the highly
competitive consumer electronics market with their relatively expensive “upgraded MP3 player.” As we now
know, the iPod has revolutionized the industry and has yielded enormous prots.
The development of the Post-It note also is a classic illustration (Hiskey, 2001). The characteristic Post-It
glue was accidentally invented in 1968 by Spencer Silver at 3M, who was trying to develop a super strong
glue for use in aerospace industry, but instead developed a glue that did not really stick, and hence was not
considered useful. The subsequent idea to use the adhesive to stick notes onto surfaces was generated by Art
Fry, a colleague of Silver’s. Unfortunately, the company’s management was not convinced and shelved the
project for several years. Only when they realized that Post-It notes were frequently used by company
secretaries did management decide in 1977 to run test sales. Although the product still was not successful
when merely advertised, this changed dramatically when 3M decided to give many samples of pads of notes
away to enable people to try the notes themselves. To their own surprise, there was a 90% reorder rate from
the businesses that had been given the samples. Post-It notes eventually became a highly successful
product.
These examples clearly show that creativity is not enough to receive (immediate) recognition or success.
Often, truly creative ideas are criticized (the iPod), rejected (the Post-It notes), or ignored (Mendel’s
seminal paper), and only become accepted and successful after time (Van Gogh) or after substantial
persistence of the creator (Rowling). Of course, this short overview is necessarily biased and incomplete;
nobody knows how many truly creative ideas or products were never published, never shown in exhibitions,
or never marketed as a commercial product. What these observations suggest, however, is that people often
fail to recognize creativity when they see it, and that people often disagree about the merits of an idea or
product. This somewhat depressing conclusion, however, would be premature for at least two reasons.
First, an assessment of people’s ability in evaluating creative ideas would require a systematic analysis of
how they judge both poor and very creative ideas. In signal detection terms, we would need to address the
hits (creative ideas that are accepted), the false negatives (creative ideas that are rejected), the correct
rejections (uncreative ideas that are rejected), and the false positives (uncreative ideas that are nevertheless
accepted). This cannot be done based on historical examples and anecdotes, because we usually do not hear
about ideas, products, or projects that were discarded for good—whether they were creative or not. Instead,
systematic research is needed in which we can actually track the quality of people’s selection as compared to
the available ideas.
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Evaluation and Selection
Second, the conclusion that creativity is generally not recognized seems at odds with other anecdotal
evidence that people do in fact often agree about the quality of an idea or product. For example, it is not
uncommon for an award jury to reach a unanimous decision, and many creative products quickly become
huge successes, implying large agreement between customers in appreciation of the product. Moreover,
creativity researchers often nd a great deal of agreement between the ratings of external judges when
these judges are asked to evaluate a product’s creativity (e.g., Amabile, 1982; Diehl & Stroebe, 1987;
Rietzschel et al., 2006; also see the following section). Thus, it would be premature to conclude that people
do not recognize creativity at all. This implies that we need to understand when people do or do not
recognize creativity and when they do or do not select creative ideas.
p. 182
A Model of Idea Recognition and Selection
The goal of this chapter is to present and organize much of the available literature on what happens to
creative ideas after they have been generated. To do so, we propose a model that integrates previous
research and may be used to guide future research eorts. Central to our model are two propositions: First,
that it is useful to distinguish between idea evaluation and idea selection, and second, that both stages
revolve around resolving the tension between originality and feasibility.
First, it is useful to distinguish between idea evaluation and idea selection. During idea evaluation, ideas are
judged on one or more dimensions, and are compared to some standard or benchmark and/or to each other
(also see Mumford, Lonergan, & Scott, 2002). Crucial to this stage (from a creativity perspective) is the
question whether people (or groups of people) are able to recognize creative ideas, and whether creative
ideas are valued or appreciated. Two errors that could occur during evaluation are, rst, that some highly
creative ideas are not recognized (e.g., evaluated as unoriginal, not feasible, or both—however, also see
Simonton, 2018), and, second, that creative ideas receive an unfavorable evaluation with regard to some
other criterion (e.g., general liking, predictions of success, or other attitudes on part of the evaluator).
During idea selection actual decision-making takes place: One or several ideas are selected for further
development or implementation. This could entail selecting ideas out of a larger idea pool, or it could be a
“go/no go” decision regarding a single available idea. In both cases an evaluation of the idea is assumed to
be fed into a decision-making process that yields a dichotomous outcome for each idea (selection versus
rejection).
An important reason for distinguishing between evaluation and selection is that idea selection is even more
challenging than idea evaluation, for two reasons. First, selection requires that multiple criteria (such as
originality and feasibility) are somehow combined into a single decision, and this can be complex if the
dierent criteria are not perfectly aligned. Second, while idea evaluation in itself is relatively open-ended,
selecting an idea has consequences and implies commitment to the selected idea. Resources will need to be
invested, and reputations and future prots are at stake: There is always the risk that an idea fails after all.
Therefore, it is to be expected that any biases or other problems in idea evaluation will be exacerbated
during idea selection.
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
The Tension Between Originality and Feasibility
The fact that creativity is risky (in that there is an unknown probability of future success) underlies the
other core proposition of our model: the tension between originality and feasibility. In creativity research,
an idea is judged to be “creative” or “of high quality” if it is both original and feasible. As explained in the
beginning of this chapter, the absence of either of these two qualities makes the idea either mundane or
irrelevant, and no longer creative. In a way, then, the creative or innovative process entails a search for ideas
or solutions that satisfy both criteria.
While most creativity research uses the dual criteria of originality and feasibility to identify “creativity,”
Litcheld, Gilson, and Gilson (2015) argue that this may not do justice to the complexity of the matter. In
their analysis of what it means for an idea to be “creative,” they address some of the dierent denitions
and operationalizations that have been used for both originality and feasibility. With regard to originality or
novelty, they note that, although “[i]deas are considered to be novel to the extent that they are uncommon
in terms of either their task or social context” (p. 242), several conceptions of novelty exist that are not
always distinguished explicitly: newness (within a particular context or domain), frequency (or rather
infrequency), and distance (the degree to which an idea is dierent from current practice—ideas with
greater distance can be considered “radical,” whereas ideas with limited distance would be considered more
“incremental”). Litcheld et al. note that these three facets of novelty are not necessarily aligned: “For
example, the idea to provide decorated facial tissue boxes for dierent seasons might not constitute a
radical idea in terms of distance, even if it is entirely new to a rm or rarely mentioned” (pp. 242–243). A
similar distinction is made with regard to feasibility, which Litcheld et al. argue is only one facet of the
broader dimension “usefulness.” While feasibility refers to the ease with which an idea could be
implemented, there is also the value of an idea, which refers to the expected eectiveness or success of the
idea. Many of the examples of failed recognition of creative ideas described earlier refer to failed recognition
of idea value. For example, the publishers who rejected the rst Harry Potter book did not believe that
publishing the books would be dicult in itself, but rather that it would be impossible to do so successfully
(i.e., yielding high prots).
p. 183
Most studies on idea evaluation and idea selection use some combination of novelty and feasibility. One
reason why it seems to be dicult to identify or select ideas that score high on both originality and
feasibility is that the two are often negatively correlated. Several authors have recently noted that these two
criteria may be seen as incompatible and represent a fundamental tension or paradox (Miron-Spektor &
Beenen, 2015; Miron-Spektor, Gino, & Argote, 2011; Zacher, Robinson, & Rosing, 2016; however, also see
Frederiksen & Knudsen, 2017). In a meta-analysis of 20 studies, Nijstad, De Dreu, Rietzschel, and Baas
(2010) found that, across these studies, the average correlation between originality and feasibility was
moderately to substantially negative (r = −.42). Although some studies have found positive correlations
between originality and feasibility (e.g., Kohn, Paulus, & Choi, 2011) or between originality and
appropriateness (e.g., Runco, Illies, & Eisenman, 2005), and it is certainly possible for ideas to be both
highly original and highly feasible, most are not. Moreover, regardless of whether this correlation happens
to exist in the idea set people are working with, people often perceive a negative correlation or even an
incompatibility between the two dimensions, and tend to focus on either of these criteria at the expense of
the other (Rietzschel, Nijstad, & Stroebe, 2010; but also see Runco & Charles, 1993).
There are several possible reasons for this negative correlation. For example, research has found that the
generation of original versus useful ideas may be triggered by completely dierent mindsets or conditions
(Bechtoldt, De Dreu, Nijstad, & Choi, 2010; Friedman & Förster, 2001; Miron-Spektor & Beenen, 2015). Thus,
Bechtoldt et al. (2010), in a study of group creativity, found that a combination of high epistemic motivation
(i.e., high willingness to think about an issue) and high cooperation facilitated originality of ideas in a
Western sample (The Netherlands), whereas it fostered feasibility of ideas in an Eastern sample (Korea).
Part of the problem may also be an artifact of people’s tendency to rely on highly accessible exemplars (e.g., Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Introduction
Ward, 1994). Many ideas that people come up with are simply minor adaptations of existing practices, and
as such are usually high in feasibility and low in originality. This can easily lead to a negative correlation, as
long as there are also some ideas with lower feasibility and/or higher originality.
However, highly original ideas are also more likely to be judged as less feasible because they involve, by
denition, a step into the unknown (also see Baer, 2012). The most original ideas are often those that are
radically dierent from existing practices or solutions (i.e., have high “distance”), which can make people
perceive them as less feasible. This same eect will occur in real-life situations, when people have to make a
decision as to which idea to choose or support. As Baer (2012) writes in his inuential paper on idea
implementation in organizations, “ideas that are useful yet novel are likely to produce uncertainty and, as a
result, are likely to be met with skepticism and hesitation[…]. Thus […]the very nature of these ideas is likely
to generate reluctance about their implementation” (p. 1103). Indeed, supporting or selecting a highly
original idea is a risky endeavor, and any factor that makes people risk-averse, or that increases their
feasibility concerns, will lower the probability of their supporting or selecting a highly creative idea.
In the following sections, we summarize research that illustrates this tension problem. We rst discuss
research on idea evaluation, and then move on to idea selection. Having summarized the available
literature, we link these ndings to some themes from the broader literature on group performance and
group decision-making. Finally, we discuss idea development as a way in which groups may be able to
overcome the tension between originality and feasibility.
Idea Evaluation by Individuals
How good are people at evaluating idea creativity? In this section, we address people’s ability to judge and
recognize creative ideas, and discuss the factors that predict or aect idea evaluation.
Although creativity is virtually always operationalized subjectively (i.e., by having people rate the originality,
feasibility, creativity, etc., of creative products), the underlying assumption is that some ideas are in fact
more creative than others, and that creativity is not merely in the eye of the beholder. If this is true, high
levels of interrater agreement are thought to indicate validity of these subjective judgments. This
assumption underlies Amabile’s (1983, 1996) consensual assessment technique (CAT), according to which
creativity is best measured by having a number of domain experts rate creative products (also see Kaufman,
Baer, Cole, & Sexton, 2008).
p. 184
Of course, reliability can only be considered a necessary requirement for validity, and not an indicator of
validity as such. Taking interrater reliability as an indication of validity raises the question whether the
raters are correct in whatever it is they agree on. Theoretically, evaluation accuracy could be assessed in
dierent ways (but see Silvia, 2008, for a thoughtful discussion of why “evaluation accuracy” may not be
not the best term). First, following the reasoning behind Amabile’s CAT and her operational denition of
creativity, creativity judgments could be said to be accurate to the extent that they correlate positively with
those from (other) experts in the eld. Second, creativity judgments could be considered accurate when they
correlate with some external criterion of idea success that depends on creativity, such as commercial
success or critical acclaim, awards given for creative contributions, and the like. Third, creativity
evaluations could be considered accurate when they correlate with some other validated measurement of
creativity; for example, subjective judgments of an idea’s originality should be correlated with the idea’s
statistical infrequency in the total idea pool (as found by e.g., Putman & Paulus, 2009, but not by Hocevar, Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Personality and Expertise
Personality
1979). Research on creativity evaluation mostly uses the rst operationalization of accuracy: Correlation
with judgments by (other) experts.
Like any form of task performance, people’s accuracy in idea evaluation depends strongly on both
motivation and ability, and these are likely to vary with individual dierences in terms of personality and
levels of expertise. Some people may be dispositionally more open toward novelty than others, and may
therefore be more motivated to carefully consider and recognize creative ideas. Further, having relevant
domain knowledge will probably be helpful in recognizing both the novelty of a product and its usefulness.
Existing research seems largely consistent with these propositions.
Silvia (2008) conducted a study on “discernment,” which refers to people’s ability to recognize their most
creative ideas. In this study, participants rst generated ideas and then selected their two most creative
responses. The goal of this research was to test whether people are discerning at all (i.e., whether a
substantial correlation between their own and experts’ judgments can be observed), and whether some
people are more so than others. Results showed a strong relation between participants’ choices and the
ratings by trained judges. Moreover, the personality trait openness to experience (the degree to which
people are curious, and enjoy puzzles, new ideas, and experiences) predicted both idea generation
performance and idea selection performance, suggesting a common factor underlying both idea generation
and discernment. Thus, people who are more open to new experiences may be more motivated to carefully
consider creative ideas, and may therefore be more accurate in their judgments.
2
The notion that idea evaluation and idea generation may depend on a common factor (like openness) is
indirectly supported by other studies on idea evaluation (Runco & Dow, 2004; Runco & Smith, 1992). In
these studies, participants rst generated ideas and then rated their own ideas for creativity; evaluation
accuracy was assessed by determining whether the ideas rated as “creative” by participants were unique or
common within the total idea pool. Although personality was not measured in these studies, the results
showed that participants who generated many original ideas were also the most accurate in their idea
ratings, which would be in line with a common trait (such as openness) contributing to both. Further, a
study by Basadur, Runco, and Vega (2000) found that the ability to produce a high number of ideas predicted
both the originality of these ideas and the ability to accurately evaluate these ideas. The underlying
predictor here was a preference for avoiding premature convergence, in other words, the ability to postpone
closure—a trait that likely correlates with openness (also see Stam, De Vet, Barkema, & De Dreu, 2013).
An alternative perspective is presented by Fürst, Ghisletta, and Lubart (2016). In their model, which they
test in two studies, idea evaluation is positively predicted by high scores on a cluster they call
“convergence,” which comprises such traits as persistence, precision, and critical sense, and correlates
positively with the Big Five factor conscientiousness. Results were generally supportive of the model;
moreover, they also found that participants’ everyday creativity was interactively predicted by generation
and selection: The positive eect of generation on creativity was stronger when selection was high as well,
underscoring that creative performance depends on both idea generation and idea evaluation. It is
notable, however, that the selection items in their study do not refer to recognizing originality; instead, the
items refer to critically evaluating one’s work and verifying ideas or correcting imperfections. As such, the
measurement may rather reect a focus on feasibility or related aspects (also see Herman & Reiter-Palmon,
2011).
p. 185
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Expertise
To summarize, although the relation between idea evaluation and personality has not (yet) been extensively
addressed, results so far suggest that the link is probably there. Moreover, looking at the results by Silvia
(2008) and Runco et al. (Runco & Dow, 2004; Runco & Smith, 1992) on the one hand, and those by Fürst et
al. (2016) on the other, it seems plausible that the tension between originality and feasibility concerns is
also found in the personality domain: High scores on openness may predict the selection of original ideas
(as well as their generation), whereas high scores on conscientiousness may predict the selection of feasible
ideas.
As mentioned earlier, the assumption underlying the CAT is that creativity is best judged by experts in the
domain of the creative task, since only experts have the necessary domain knowledge to recognize whether
an idea is original (i.e., dierent from what is commonly done or has been done in the eld) and feasible or
appropriate (i.e., ts the domain and makes a sensible contribution to it). Several studies have addressed
this issue, and generally nd that expertise indeed inuences creativity judgments.
Kaufman, Baer, Cole, and Sexton (2008) had poems rated by both experts (published poets) and novices
(college students with no particular poetry expertise). All raters were asked “to use their own personal sense
of what is creative” (p. 174). Results showed that experts rated the poems as less creative than novices, and
that the correlation between expert and novice ratings was signicant, but small to medium in magnitude.
Moreover, reliabilities of the expert versus novice ratings were quite dierent, with experts showing much
higher interrater reliability than novices. These results could be seen as support for the CAT assumption
that it is better to use expert raters, at least in terms of interrater reliability.
In a more recent study, Kaufman, Cropley, Baer, Reiter-Palmon, and Sinnett (2013) also looked at the
evaluation performance of quasi-experts: People with more domain experience than novices, but without the
professional experience or standing of experts. With regard to the expert–novice distinction, results were
consistent with Kaufman et al. (2008): Experts showed higher interrater reliability than novices, and
correlations between expert and novice ratings were low to moderate. The results for quasi-experts were
less consistent: Quasi-experts sometimes, but not always performed as accurately as experts. On the whole,
however, the results clearly show the importance of using judges that have at least a moderate amount of
domain expertise.
Other studies conrm the general notion that expertise aects the evaluation of creative ideas (also see
Dailey & Mumford, 2006). Onarheim and Christensen (2012) conducted a study on the idea-screening
process in the medical industry, and found that the correlation between the evaluations of inexperienced
and experienced employees was quite high; however, the authors also note that inexperienced employees
were more likely to base their evaluations on visual complexity (a potentially misleading heuristic). In a
study on expertise and product evaluation, Moreau, Lehmann, and Markman (2001) found that the impact
of expertise on the evaluation of creative products depended on the innovativeness of the product. For
“continuous” innovations (i.e., more incremental innovations), higher expertise was associated with better
comprehension of the product and its possible benets, and hence with higher preferences. The opposite
was true for “discontinuous” (more radical) innovations: There, higher expertise was associated with lower
comprehension, and more resistance to the product (also see the chapter 3 by Paulus, van der Zee, &
Kenworthy, this volume).
In sum, factors that inuence people’s openness to new ideas (e.g., the personality trait openness to
experience) and their ability to adequately judge contributions to the eld (expertise) clearly matter when it
comes to idea evaluation. Some evidence additionally suggests that the inuence of expertise may depend
on features of the product, such as the radicalness of an idea: Experts may be more capable of recognizing
the creativity of incremental as opposed to more radical ideas. This may make sense, because more radical Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Creative Forecasting
The Motivation to Recognize Creativity
ideas depart substantially from existing preferences, and may be hard to evaluate even for people with high
levels of expertise.
Another way of thinking about idea evaluation is in terms of the predictions people make about an idea’s
success. In fact, Mumford, Lonergan, and Scott (2002), who proposed a model of idea evaluation, state
that such predictions are the essence of creative idea evaluation: “A new idea cannot be evaluated as an
entity unto itself. Instead, evaluation occurs by appraising the idea in context. […] [E]valuation will, most
often, be based on some anticipated […] outcomes of idea implementation” (pp. 21–22). Indeed, especially
in organizational contexts, the expectations that people have about the likelihood that an idea will be
successful (e.g., will solve a problem, will become a commercial success) will drive their evaluations or their
adoption decisions (cf. Litcheld et al., 2015). As we have seen in our discussion of real-world examples, this
creative forecasting (Berg, 2016; Byrne, Shipman, & Mumford, 2010; Dailey & Mumford, 2006) does not
always go so well: Many ideas that get rejected because they are expected to fail turn out to be very
successful later on.
p. 186
A recent set of studies by Berg (2016) looked at the role of expertise and evaluation criteria during creative
forecasting. These studies revolved around the role that people have in the organization. For example, in a
eld study among circus professionals, Berg found that, although both managers and creators
underestimated the success of new acts, managers did so more strongly than creators, especially for highly
novel ideas. However, the better forecasting performance of creators disappeared when it came to
forecasting the success of their own (rather than somebody else’s) ideas: Managers actually performed
better than creators forecasting the success of the creator’s own ideas. Berg argues that the manager role,
with its exclusive focus on decision-making and evaluation, eectively overrides the divergent processes
inherent in the creator role, which in turn decreases evaluation accuracy. A potential interpretation is that
for managers, implementation concerns are more salient than for creators and this makes them reluctant to
support truly original ideas.
Other research has focused not so much on people’s ability to recognize creative ideas or the role of
expertise, but rather on the motivational processes and biases that may come into play in idea evaluation. An
overarching theme that arises from this research is, as mentioned before, the tension between originality
and feasibility: In general, people tend to be more open toward creative ideas (i.e., evaluate them more
positively or accurately, and are more likely to adopt them) when they are open to the uncertainty and risks
that are associated with creative ideas.
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Regulatory Focus
One theory that directly relates to people’s willingness to take risks versus playing it safe, and that has been
studied extensively in the eld of creativity, is Higgins’s (1997) regulatory focus theory. Regulatory focus
theory states that people have two distinct self-regulatory systems, one of which may be dominant at any
given moment. The promotion system is concerned with growth, nurturance, and the realization of
ambitions, whereas the prevention system is concerned with safety, security, and the fulllment of
responsibilities. When people are in a promotion focus, they tend to be concerned with gains (versus
nongains), whereas people in a prevention focus tend to be concerned more with losses (versus nonlosses).
This has implications for risk-taking (Crowe & Higgins, 1997): In a promotion focus, people are more likely
to adopt a risky bias and to avoid errors of omission, whereas people in a prevention focus are more likely to
adopt a conservative bias and to avoid errors of commission. The implications for creativity are clear: All
other things being equal, it is to be expected that people in a promotion-focused state will show higher
levels of creativity, as indeed has repeatedly been found to be the case (e.g., Baas, De Dreu, & Nijstad, 2008;
Bittner, Bruena, & Rietzschel, 2016; Friedman & Förster, 2001; Herman & Reiter-Palmon, 2011; Rietzschel,
2011; however, also see Baas, De Dreu, & Nijstad, 2011). More relevant to the current chapter, however,
regulatory focus has also been found to aect idea evaluation.
In particular, Herman and Reiter-Palmon (2011) found that regulatory focus exerted dierent eects on
idea generation and idea evaluation. All ideas generated by participants were rated by external judges and by
participants themselves on originality and feasibility (labeled “quality” by the authors, i.e., the degree to
which the ideas was “logical, coherent, well thought out, and workable” [p. 16]). For idea generation, as
found earlier, participants’ promotion-focus state positively predicted their creative performance (i.e.,
originality levels of their ideas as rated by expert judges). For idea evaluation, however, the accuracy of
participants’ evaluation (measured as the squared dierence between their own and experts’ ratings)
depended on the creativity dimension. Participants in a promotion focus were more accurate when it came
to evaluating idea originality, but participants in a prevention focus were more accurate when it came to
evaluating idea feasibility. Herman and Reiter-Palmon (2011) suggest that people in a promotion focus
may be so focused on novelty and growth that they neglect considerations of feasibility or appropriateness,
resulting in a less accurate evaluation for those aspects (and vice versa for people in a prevention focus).
These results are similar to those obtained by Fürst et al. (2016), who found that participants’ tendency to
critically evaluate and correct their own ideas was predicted by the personality trait “convergence”
(persistence, precision, and critical sense) and conscientiousness, all of which t well with the vigilant and
careful processing mode associated with a prevention focus.
p. 187
These results are further supported by Zhou, Wang, Song, and Wu (2017), who also showed that regulatory
focus aected participants’ recognition of novelty and creativity, such that participants were better at
recognizing novelty and creativity when they had a strong promotion focus (both as compared to a weak
promotion focus and as compared to a strong prevention focus). Recognition of novelty and creativity were
also enhanced by an innovation-centered organizational culture. Similarly, De Buisonjé, Ritter, Bruin, Ter
Horst, and Meeldijk (2017) found that participants who were brought into a promotion-focused state
(combined with a positive mood state and a self-armation manipulation) selected more creative ideas
than participants in a control condition.
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Construal Level
A Bias Against Originality?
Mueller, Wakslak, and Krishnan (2014) adopted a construal level perspective to examine idea evaluation.
Construal level theory (e.g., Trope & Liberman, 2010) states that people can represent objects (including
ideas) or events (including decisions) on dierent levels of abstraction, and that this construal level is a
function of the psychological distance between the person and the object or event. Low-level construals are
concrete and detailed, and focused on “how” considerations (e.g., how is a certain event going to take
place). High-level construals are abstract and global, and focused more on “why” considerations (e.g., why
an event is going to take place). These dierences in representation aect a variety of cognitions and
behaviors, including creativity: Creativity seems to benet from a high construal level because of the
abstract mindset and broad attentional scope associated with it (e.g., Förster, Friedman, & Liberman, 2004;
Jia, Hirt, & Karpen, 2009 Schimmel & Förster, 2008).
Mueller et al. (2014) argued that a low-level construal, with its emphasis on practicality and its narrow
attentional scope, should be associated with less eective idea evaluation than a high-level construal. In
three studies, Mueller et al. (2014) found that participants who were brought into a high construal level
state evaluated highly creative ideas as more creative than participants in a low construal level. Moreover,
this eect was specic to highly creative ideas: For noncreative ideas, no such eect was observed,
suggesting that the eect reects accurate evaluation (or recognition) of creativity, rather than a general
tendency to evaluate ideas as more creative.
The above studies suggest that originality is better recognized when in a promotion focus state or when
adopting a higher construal level, because these motivational states encourage risk-taking and reduce the
focus on implementation concerns. However, Mueller, Melwani, and Goncalo (2012) also proposed that even
when people say they value creativity, they may nevertheless have an unconscious bias against originality,
and that this bias is especially likely to be activated when people feel the need to reduce uncertainty
(because creative ideas are uncertain and risky). They found support for this hypothesis across two studies:
In both cases, when the motivation to reduce uncertainty was manipulated, participants displayed a
stronger implicit bias against originality (as measured with the Implicit Association Test), but not a
stronger explicit bias. Moreover, they found that this implicit bias interfered with people’s ability to
recognize creativity. The eects of uncertainty also t with the research discussed earlier on promotion and
prevention: It is plausible that people with a strong prevention focus are also more motivated to reduce
uncertainty (because uncertainty threatens their need for safety), and hence are less able to recognize or
value originality.
In line with the general notion that people may be biased against originality, Blair and Mumford (2007) had
participants rate and select ideas in a project funding task, and found that participants generally preferred
ideas that, among others, were consistent with social norms, were likely to quickly yield desired eects,
were easy to understand, and had broad benets for society. Moreover, people actually tended to reject ideas
that were highly original or that were risky: These ideas were evaluated less positively, and were less likely
to be recommended for further development. As Blair and Mumford conclude, “Perhaps the most salient
conclusion to be drawn from this study is the undeniable disdain for risky and original ideas” (p. 215).
Interestingly, the preference for ideas conforming to social norms and yielding quick results was
strengthened under time pressure (also see Chirumbolo et al., 2004).
p. 188
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
The Tension Between Originality and Feasibility in Idea Evaluation
Introduction
Taken together, the results on regulatory focus, uncertainty, and construal level clearly demonstrate the
tension between originality and feasibility. People are less likely to recognize or support highly creative
ideas if their motives are primarily aimed at safety, certainty, and feasibility, or when the problem is
psychologically close. Factors that enhance the willingness to take risks and to take an open, exible
approach toward the problem or task also enhance the recognition of creativity and the willingness to
support it.
From an applied perspective, this presents something of a challenge, as creative ideas are, of course, usually
sought with an eye to application. Although originality in itself may not always be particularly important
when solving problems (as long as one nds a solution that works), it is exactly when conventional
solutions no longer work that creative ideas are most desperately wanted (cf. Osborn, 1957). Paradoxically,
the sense of urgency that may arise in these situations can easily bring people to reject highly creative ideas,
even though these may hold the potential for solving the problem.
Idea Selection by Individuals
Idea evaluation, of course, is only an intermediate step in the creative or innovative process. In the end, one
or several ideas need to be selected. In this section, we review research on idea selection performance,
beginning with idea selection by individuals.
The eectiveness of idea selection will partly depend on people’s ability to recognize good ideas. However,
idea selection additionally involves combining, reconciling, or making a trade-o between dierent criteria
in order to arrive at a single decision. For example, people could focus on idea originality, radicalness,
feasibility, usefulness, eectiveness, eciency, market potential, or any of a number of other criteria (e.g.,
Dailey & Mumford, 2006; Frederiksen & Knudsen, 2017). Which criterion receives more weight will, among
other things, depend on people’s goals and motivation. In any case, the negative correlation between
originality and feasibility can make it dicult for people to make a selection that satises both criteria.
Moreover, as briey discussed earlier, selection implies some degree of commitment: Actually selecting an
idea means that certain resources (such as time and money) will be invested in the development,
implementation, and marketing of the idea (also see Baer, 2012). Having made a wrong choice will become
ever more costly as the innovation process continues, and one’s reputation or future prospects (either as an
individual, a project team, or an organization) can be seriously damaged by having supported the wrong
idea, or by having rejected an idea that subsequently brings a competitor a lot of success. Thus, the stakes
are higher for selection than for evaluation, and this can easily strengthen biases or introduce new ones. All
in all, it seems likely that the tension between originality and feasibility will exert even stronger eects on
selection than on evaluation. Several studies have looked at idea selection, and we summarize this research
in what follows.
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Empirical Evidence
The Tension Between Originality and Feasibility in Idea Selection
A few studies have addressed idea selection on the individual level. For example, Rietzschel et al. (2010)
noted that it may be hard to select the best ideas among an idea set, especially when there are many ideas to
choose from (also see Reiter-Palmon & Arreola, 2015). They tested whether selection performance could be
improved by giving participants explicit selection criteria, and by having participants engage in a
prescreening of ideas before the nal selection. While prescreening did not aect idea selection, selection
criteria did, depending on the criteria given. Telling participants to select ideas that were both original and
feasible did not lead to better idea selection (as compared to simply telling participants to select “the best”
ideas), although it did lead to lower satisfaction. Telling participants to select creative ideas (without further
specifying what this meant) led to the selection of ideas that were rated as higher in originality, but lower in
eectiveness, and again led to lower satisfaction. Rietzschel et al. (2010) conclude that participants’ strong
focus on feasibility and desirability of ideas stands in the way of their selecting more original ideas, as
originality is often perceived to be negatively correlated with feasibility (also see Blair & Mumford, 2007).
In another study, Rietzschel, Nijstad, and Stroebe (2014) combined a manipulation of problem scope (a
broad versus a more narrow topic) and creativity instructions (telling participants to focus on the creativity
of their ideas, during both generation and selection—the other participants were instructed to focus on
the personal relevance of their ideas). Problem scope had no eect on selection performance, although a
narrow problem did increase idea originality. Creativity instructions had a similar eect as the selection
criteria in Rietzschel et al. (2010): Participants with creativity instructions selected more original ideas, but
were less satised with their selection. Participants with relevance instructions failed to perform above
chance level in their selection, demonstrating that idea selection is even more problematic than idea
evaluation.
p. 189
In another line of research, Ritter, Van Baaren, and Dijksterhuis (2012) looked at the eects of unconscious
thought in the creative process, and found that participants who were given the opportunity to engage in
unconscious thought (similarly to the “incubation” stage sometimes mentioned, e.g., Wallas, 1926) selected
ideas of higher creativity than participants who did not engage in unconscious thought. One reason why the
opportunity for unconscious thought might benet idea selection is that it helps people loosen the implicit
assumption that ideas are either original or feasible (see the earlier discussion). Research on insight
problems suggests that relaxing implicit assumptions is key to successful creative problem solving
(Sternberg & Davidson, 1995), and this may also hold for creative idea selection. Conscious deliberation may
therefore not always be the best way to arrive at the selection of creative ideas, because participants may
rely too much on their assumptions regarding originality and feasibility. Thus, for example, Zhu, Ritter,
Müller, and Dijksterhuis (2017) found that participants selected more original ideas when they were
instructed to make their selection intuitively rather than deliberately.
The results on individual idea selection suggest that eective idea selection is highly problematic, and
probably even more so than evaluation, mostly because individuals tend to focus on feasibility at the cost of
originality. The nding that participants were less satised with their selection when instructed to select
creative or original ideas illustrates the problem: People do indeed seem to have a persistent bias against
original ideas, and even though they may be able to recognize and select creative ideas, this does not imply
actual support for such ideas. This is likely to be especially problematic within the organizational context
(also see Baer, 2012).
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Introduction: Individuals Versus Groups
Idea Evaluation and Selection in Groups
The research on individual idea evaluation and selection suggests that individuals often focus on feasible
ideas and ignore originality, and that the main reason is that originality is associated with uncertainty, risk,
and implementation concerns. We now move to the group level, and explore the performance of groups
during idea evaluation and selection, and consider whether groups are better at idea evaluation and
selection than individuals.
Studies on group judgment and decision-making suggest that groups potentially outperform individuals on
these kinds of tasks, but also that this does not always happen and that groups at times even perform worse
than individuals. The literature on decision biases, for example, suggests that groups are sometimes more,
sometimes less, and sometimes equally susceptible to decision biases and errors as individuals (Kerr,
MacCoun, & Kramer, 1996).
Making a group decision often largely depends on the preferences of individuals. Social decision scheme
(SDS) theory (Davis, 1973; also Stasser, 1999) suggests that group decisions are a consequence of group
composition in terms of member preferences, and an SDS or decision rule that transforms individual
preferences to a group decision. Dierent SDSs are possible, but most relevant here are “majority wins” and
“truth wins.” In majority wins, the group will choose the option that is endorsed by the majority of
membership. In truth wins the assumption is that one alternative is better than the others (i.e., is the
“truth”), and that the group will choose it as long as at least one member is in favor of that alternative.
Whether groups adopt a majority wins or a truth wins SDS depends on the nature of the decision-making
task (Laughlin & Ellis, 1986). In tasks that mainly depend on subjective judgments (so-called judgmental
tasks), groups will often adopt a majority wins decision rule. In tasks in which one answer is demonstrably
correct (so-called intellective tasks), as in some problem-solving tasks (e.g., arithmetic problems), groups
are more likely to use a truth wins SDS. Groups will perform better than the average individual when they
use a truth wins SDS, because only one correct group member is needed to convince the whole group to
adopt the “correct” response, and the probability of at least one group member identifying the correct
response is of course higher in groups than in individuals. However, given the considerable diculty in
identifying and selecting creative ideas, or in predicting their success, choosing ideas is more like a
judgmental task, and groups are likely to rely on majorities when performing this task.
p. 190
A robust prediction and nding in the SDS literature is that decision biases among individuals will be
aggravated in groups when groups use a majority wins SDS (see e.g., Kerr et al., 1996; Stasser, 1999). The
reason is that if a certain bias is present among the majority of individuals, it becomes likely that groups
also contain a majority of members who have this bias. Because this majority will prevail, groups and their
collective decisions will often be subject to the bias as well. For example, if at the individual level 70% of
people are subject to a certain bias, and six individuals are randomly selected to form a group, the likelihood
that the group contains a majority (i.e., four or more members) of members who have the bias is about 75%
(and in about 17.5% of the cases, half of the group members will have the bias). Thus, (at least) 75% of
groups will have the bias, as compared to 70% of individuals. The implication for idea selection may
therefore be that groups, even more strongly than individuals, have a bias against originality, because
individual level biases are aggravated in groups.
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Empirical Evidence
The previous arguments suggest that groups will not do better than individuals when it comes to idea
selection, and that the bias against original ideas may even be stronger in groups. Group idea selection has
been investigated in several studies. Some of the rst studies on group idea selection were done in the
context of brainstorming. The goal of a brainstorming session is to come up with as many creative ideas as
possible, with the assumption that a higher availability of high-quality ideas should also facilitate the
selection of high-quality ideas. One goal of these idea selection studies therefore was to test whether this
was the case: Do groups or individuals who generate more ideas also select better ideas? Another goal of this
research was to study the eects of group interaction per se on idea selection (i.e., whether groups would be
better or worse at selecting creative ideas than individuals).
The rst paper to address this question was published by Faure (2004), who compared nominal and
interactive groups on a brainstorming task followed by an idea selection task. Nominal groups are groups
“in name only” and consist of individuals who brainstorm alone, and whose nonredundant ideas are pooled
(i.e., their pooled ideas constitute the “group” output). Having generated ideas either individually (as part
of a nominal group) or as an interactive group, participants rst made an individual preselection from the
entire group’s output, after which the entire group made a selection from these preselected ideas. In line
with other research (see Stroebe, Nijstad, & Rietzschel, 2010, for an overview), Faure (2004) found that
nominal groups generated more ideas and more original ideas than interactive groups in the rst stage.
However, there was no dierence in terms of idea selection in the second stage: Despite the availability of a
higher number of original ideas in the nominal groups, this did not lead to the selection of more original
ideas. This is consistent with what has been found at the individual level: If idea selection performance is
poor (and often not better than chance) one would not expect that the availability of more high quality ideas
per se matters. Thus, this study seems to suggest that group idea selection is not much better than
individual idea selection.
In the Faure (2004) study, idea selection always took place in a group context (after either individual or
group idea generation). Rietzschel et al. (2006) conducted a similar experiment, with participants rst
generating and then selecting ideas. However, in this study the nominal-interactive dierence was
maintained in the selection phase: Participants in the nominal group condition selected ideas on their own,
and the originality and feasibility of their rst choices were averaged across group members. Similarly to
Faure (2004), Rietzschel et al. (2006) found no dierence between interactive and nominal groups in terms
of the quality of the selected ideas, even though nominal groups were more productive. Moreover, when
testing the quality of the selected ideas against the quality of the generated ideas, it turned out that these
were not signicantly dierent; in other words, selection performance was not better than chance, either in
nominal or in interactive groups. These results suggest, perhaps even more strongly than the results of
Faure (2004), that groups are not better in idea selection than individuals.
Putman and Paulus (2009) also conducted an experiment on idea generation and selection in nominal and
interactive groups. Similarly to the procedure used by Faure (2004), idea selection took place in interactive
groups. As usual, nominal groups were more productive than interactive groups. However, in contrast to
Faure (2004) and Rietzschel et al. (2006), this study found that nominal groups also selected more original
ideas. In line with Rietzschel et al. (2006), however, Putman and Paulus (2009) also found that selection
performance was suboptimal: On the whole, the mean originality of the selected ideas was not higher
than that of the generated ideas.
p. 191
A somewhat dierent result was reported by Larey (1994; also see Larey & Paulus, 1999), who found that
interactive groups were more accurate in identifying their most creative ideas than individuals, especially if
the groups were composed of people with a strong preference for group work. Thus, this research did show a
group advantage; however, the task used in this research was more of an evaluation than a selection task: Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
participants were asked to identify the most creative ideas. This suggests that, in line with other research
discussed above, idea evaluation may be somewhat less dicult than idea selection, even or especially in the
group context.
Several other, nonbrainstorming studies have also looked at idea selection in groups. For example, Girotra,
Terwiesch, and Ulrich (2010) studied the eects of two kinds of “group structure” on the process of idea
generation and idea selection. Participants rst generated and then selected ideas on a variety of problems.
They did this either as a group throughout the whole process, or in a hybrid structure, where participants
rst generated ideas individually (and selected their own favorite ideas) and then came together as a group
to continue the idea generation and subsequent idea selection. The researchers found that groups in the
hybrid condition (alternating between individual and group work) were more productive. When it came to
selection, groups in the hybrid condition selected ideas that were rated as more attractive for potential
customers, but there was no dierence in rated business value of the selected ideas in the two conditions.
Moreover, the authors point out that groups in all conditions performed poorly in terms of selecting their
best ideas.
Perry-Smith and Co (2011) addressed the role of collective mood states in groups. While the relation
between aect and idea generation has been studied extensively (see e.g., De Dreu, Baas, & Nijstad, 2008, for
a review), virtually all of this research focused on individual-level aect and creativity, and had not looked
at idea selection. In a study among student workgroups, Perry-Smith and Co (2011) assessed idea
generation and idea selection, as well as mood states during the group project. In contrast to the common
assumption that productivity should enhance the selection of good ideas (because more good ideas should
be available; see earlier discussion), the correlation between productivity and the novelty of the selected
ideas was actually negative: Groups that generated more ideas, tended to select ideas that were less novel
(also see Stam et al., 2013). With regard to group aect, the results suggested that dierent activities
required dierent mood states. Groups generated more ideas when their collective mood was positive and
activating (e.g., happy, elated; also see De Dreu et al., 2008). For idea selection, however, the results were
more complex: Selection of original ideas was facilitated by a positive, nonactivating group mood (calm and
relaxed, but not drowsy or bored), but selection of feasible ideas was facilitated by activating group moods
that were both positive and negative in valence (e.g., both happiness and anger), supposedly because these
mood states made groups constructively critical.
In a study with teams of design students, Toh and Miller (2016a) found that teams selected more novel
design concepts when the members scored high on conscientiousness, agreeableness, and tolerance for
ambiguity. Furthermore, success at the generation of novel concepts did not predict selection performance
(also see Toh & Miller, 2016b). Toh and Miller point out that although design students are trained to be
aware of the importance of creativity in design, this apparently does not necessarily lead to their selecting
their most creative concepts.
In a recent eld study on idea selection by organizational research and development (R&D) panels,
Criscuolo, Dahlander, Grohsjean, and Salter (2017) hypothesized and found that panels were most likely to
select and fund ideas of moderate (as opposed to low or high) originality. However, when panels had a high
workload (e.g., have more proposals or more extensive proposals to consider), the maximum point of this
inverted U shape decreased; in other words, they were less likely to fund even moderately original ideas,
preferring relatively unoriginal ones instead. In contrast, and in line with one of the arguments for using
groups to select ideas, panels with high expertise diversity preferred more original ideas. Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Conclusion and Discussion
On the whole, then, groups do not seem to do very well when it comes to selecting creative ideas, often not
even better than chance level and also not better than individuals. While most studies on idea evaluation do
show a positive correlation (ranging from modest to substantial) between participant and expert ratings,
this discernment (Silvia, 2008) does not seem to lead to eective selection, even though work by Larey
(1994) suggests that groups may be somewhat better at recognizing creative ideas. Moreover, group
creativity in the sense of idea generation does not seem to be strongly related to selection performance:
Generating more ideas or more good ideas does not guarantee that better ideas are eventually selected.
Whether groups are even more biased than individuals, as can be predicted based on SDS theory, has not
been systematically tested; in fact the only study that actually directly compared idea selection by
individuals and groups (Rietzschel et al., 2006) has found no dierence in selection eectiveness. More
importantly, perhaps, only few studies have looked at how selection eectiveness in groups may depend on
other factors or how it can be improved. The existing work suggests that collective mood (Perry-Smith &
Co, 2011) and personality of members (Toh & Miller, 2016a, 2016b) play a role, although it may be
premature to draw rm conclusions.
p. 192
Nonetheless, based on the previous work on idea evaluation and selection by individuals, it is possible to
generate some hypotheses. In the introduction of this section, we emphasized SDS theory, and assumed that
a group decision would be the consequence of group composition in terms of member preferences and the
SDS that is adopted by the group. According to this perspective, there would be two ways to improve
selection eectiveness of groups. The rst would be to change individual preferences, and make individuals
less biased in their judgments of creative ideas. The second would be to change the SDS that is used by the
group.
First, in the sections on individual idea evaluation and selection, we noted that several psychological or
motivational traits and states, such as high openness to experience and tolerance for ambiguity, high
promotion focus, and an abstract construal level, improve individuals’ recognition of creativity as well as
their evaluation accuracy (mainly in terms of originality). Such motivational states often have an equivalent
at the group level. For example, research on regulatory focus suggests that groups may develop a shared
promotion or prevention focus (e.g., Levine, Higgins, & Choi, 2000). Thus, a group climate that is
characterized by a collective promotion focus (or a collective high construal level) may lead members to
better recognize especially originality, which may improve selection performance with regard to originality.
Consistent with this proposition, it has been found that a collective promotion focus may facilitate idea
promotion in groups, an activity that rst requires groups to select ideas that they want to promote to
others (Rietzschel, 2011). Similarly, groups may be more or less open to new experience or tolerant of
ambiguity, and to the degree that this is the case, one would expect better selection performance.
Second, the work by Rietzschel et al. (2010, 2014) at the individual level suggests that selection performance
may be improved by giving specic selection criteria. It may be the case that such explicit criteria are
especially helpful in groups, because they make it more easily defendable that certain ideas should be
chosen. Thus, when selection criteria explicitly emphasize originality as an important criterion, a member
in favor of selecting a highly original idea may be more able to eectively argue why this idea should be
chosen. In that sense, selection criteria may inuence the SDS adopted by the group (e.g., rather than a
majority, a minority may be sucient to convince the group to adopt an original idea).
To conclude, when left to their own devices, groups are not particularly eective when selecting creative
ideas. However, there are potential ways to improve selection eectiveness in groups, including measures
that make individual members more open to original ideas, and giving more explicit selection criteria.
Evidently, more research is called for to shed further light on this important topic.
Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
Implications and suggestions for future research
Idea development
Discussion
We started our review of the literature from the assumptions that, rstly, it is useful to distinguish between
idea evaluation and idea selection; secondly, that the ability and motivation to eectively evaluate and
select ideas is dependent upon both personal and situational characteristics; thirdly, and most importantly,
that the tension between originality and feasibility is the main threat to eective idea evaluation and
selection. In line with these assumptions, we have seen that idea evaluation and idea selection are dierent
in that people are generally able to recognize creative ideas (although not always to the same extent), but
apparently are not very willing to select them unless explicitly instructed to do so. More research has been
done on idea evaluation than on idea selection, and overall this research suggests that people are better at
recognizing creative ideas when (a) they are good at generating ideas, (b) they are open to new experiences
and uncertainty, (c) they adopt a global mindset and are willing to take risks, (d) have at least moderate
expertise within the domain, and (e) adopt a perspective or role that is more “creator”-like than
“manager”-like. Results for idea selection show that, on the whole, (a) both individuals and groups perform
poorly at idea selection, (b) this seems to be largely due to a focus on feasibility, (c) explicit creativity
instructions can stimulate creative idea selection, but at the cost of lower satisfaction, and (d) factors
that stimulate idea generation do not necessarily enhance idea selection.
p. 193
The most important challenge that arises from the studies discussed here is the tension between originality
and feasibility. Both the recognition and selection of creative ideas strongly depend on people’s willingness
to take the risk of considering an idea that may seem unfeasible at rst. A clear implication, then, is that this
openness to originality should be enhanced, and that groups and teams interested in better creative
performance need to be alert for factors (both individual and contextual) that may decrease this openness.
Thus, for example, teams operating under circumstances that are likely to engender a prevention focus or a
low construal level (e.g., when an organization faces decreasing sales and needs to innovate quickly in order
to survive) will nd it dicult to recognize and select creative ideas. Similarly, groups should try to
organize their collective creative eorts in such a way that the recognition and selection of creative ideas is
stimulated as much as possible.
One particularly promising way to do this might be to adopt a dierent approach towards idea generation
and selection, such that the perceived risk or threat inherent in supporting or selecting highly original ideas
is reduced. We will discuss this approach, idea development, below.
So far, we have, in line with most of the existing research, worked from the assumption that idea generation
and idea selection are two separate activities (e.g., Rietzschel et al., 2006). Of course, in reality the two are
often mixed, and projects may iterate repeatedly between these stages. For example, while discussing or
selecting ideas that have been generated previously, an idea may be changed (e.g., a highly original yet
unfeasible idea may be adapted to be more feasible), two or more ideas may be combined, or wholly new
ideas may arise, possibly because of cognitive stimulation resulting from exposure to multiple creative ideas
(Kohn, Paulus, & Choi, 2011), or because the problem is redened. Such idea development may lead to higher
creative outcomes than merely making a choice from among the previously selected ideas.
For example, in a study with teams of design students, Toh and Miller (2015) found that teams selected
more creative ideas to the extent that they actively reected on ideas and their components, and used this
reection to generate new ideas during the discussion. Toh and Miller conclude that this runs counter to the Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
more traditional practice of generating ideas before discussing and selecting ideas, and that encouraging
teams to let their ideas evolve during the design process should lead to the selection of more creative
products. This is in line with results obtained by Hao, Ku, Liu, et al. (2016), who found that participants who
were given the opportunity (halfway through the generation task) to reect on the originality of the ideas
generated thus far, subsequently generated more original ideas than participants who did another,
unrelated task. Thus, some reection on the originality of one’s ideas may enhance subsequent idea
generation, and as such can be a useful ingredient for the idea development process (also see Harvey and
Kou, 2013). Interestingly, Toh and Miller (2015) also found that teams tended to focus their discussion more
on feasibility than on novelty, which suggests that even in an idea development approach a premature focus
on feasibility is something to actively avoid.
Lonergan, Scott, and Mumford (2004) argue that revision is an integral part of idea selection. They adopt a
model of idea evaluation (Mumford et al., 2002) that begins with forecasting (see above) of the likely
outcomes of an idea, followed by appraisal of the projected outcomes against a set of criteria or standards.
Idea revision is the next step, because only rarely will an idea be ready for implementation “as is”. However,
beside the fact that most ideas will indeed need to be revised before they can be implemented, we argue that
revision of ideas oers the opportunity to make ideas t criteria that could otherwise lead to (premature)
rejection (Frederiksen & Knudsen, 2017). In other words, the revision or development of ideas (as opposed
to a dichotomous accept/reject decision) might be the best way to overcome the tension between originality
and feasibility. For example, a highly original yet unfeasible idea could be adapted to make it more feasible,
or perhaps some components of the idea can be developed further as new ideas. Taking an idea development
approach may give people the opportunity to postpone rejection of ideas, because highly original ideas do
not represent as much of a risk (yet).
Litcheld et al. (2015) address idea development from the perspective of gaining support for ideas in their
initial form, thus making sure that the idea survives long enough within the organization to make it to
implementation, or to spark new ideas that end up being implemented. Dierent kinds of creative ideas,
they argue, will require dierent development trajectories because they are likely to run into dierent
problems. While a common approach to the development of “radical ideas” (which are high on novelty and
value, but perceived to be low on feasibility) is “to provide some sort of safe haven for working out the
feasibility challenges” (p. 251), Litcheld et al. argue that these interventions “merely buy time”, and that
their successful development may rather require seeking out early idea champions who are primarily
focused on the idea’s value. In contrast, the further development of “disruptive ideas” (high on novelty and
feasibility, low on value) might be better served by analyzing the value of the idea or the (small) market
segment where that value might exist, actively seeking out “some corner where they create value now and
by virtue of their feasibility can be implemented immediately” (p. 253), rather than developing the idea in
the background in the hope that some sort of value will eventually emerge. An example may be Google Glass,
a wearable device that was meant to be both convenient and stylish, but failed as a consumer product (e.g.,
Naughton, 2017) because people tended to nd it both creepy (e.g., because the wearer might be lming
stealthily) and silly-looking. However, the device later turned out to be somewhat more successful in
industrial settings, for example as worn by factory workers who needed to be able to look up or share
information while working with their hands (Levy, 2017). Thus, while Google Glass demonstrated low value
in terms of the general consumer market, it turned out to have much higher value within one specic niche.
p. 194
There is reason to suppose that groups are especially suited for the idea development or revision process,
because it is here that the ability to combine multiple perspectives becomes really worthwhile. Group idea
generation tends to be ineective because of production blocking (Nijstad & Stroebe, 2006), and unaided
group decision-making often is suboptimal, because groups often fail to make use of all the available
information, tend to prematurely work towards consensus, succumb to conformity pressure, et cetera.
Research by Harvey (2013) also shows that idea revision and development may be a challenge for (diverse) Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
groups. However, if groups do manage to use and combine their multiple perspectives, they might well
outperform individuals in the revision stage (also see Larey & Paulus, 1999). Recent support for this notion
was obtained by McMahon, Ruggeri, Kämmer, and Katsikopoulos (2016). In two studies, they compared the
creative performance of groups, nominal groups, and individuals. In the rst study, where participants
generated, selected, and developed ideas, no dierences between conditions were found on the quality of
the nal creative product. However, in a second study that focused more specically on idea development,
participants were provided with a single pre-generated idea to develop further. In this study, groups
outperformed individuals, by producing developed ideas that were rated higher on marketability, fun, and
overall quality. Qualitative analysis of the discussions showed that groups addressed a wider range of issues
and perspective than individuals (who were instructed to take notes of their thoughts and considerations
during idea development).
Conclusion
Idea evaluation and idea selection remain a challenge, both for practitioners and researchers. From a
practical perspective, it is clear that evaluation and selection are stages where the creative process is
seriously at risk: Creative ideas can easily go unnoticed or be rejected. From a research perspective,
evaluation and (especially) selection remain understudied: Although some patterns are beginning to emerge
(especially regarding the tension between originality and feasibility), we still know far too little about these
crucial stages in the creative process. What we do know is that people have a bias against original ideas,
because these ideas by denition imply risk and uncertainty. The main challenge, for research and practice,
therefore, is to mitigate this bias—in individuals and in groups—so society benets more from the
creativity of its members.
Notes

1. The examples we discuss here were found in general online sources such as Wikipedia, although some cases were
   discussed more elaborately elsewhere, such as https://www.theguardian.com/books/2016/mar/25/jk-rowling-harrypotter-posts-letters-of-rejection-on-twitter and http://www.litrejections.com/best-sellers-initially-rejected/.
2. We discuss this study in the section on idea evaluation, because the “selection” in this study basically came down to
   identifying two ideas that scores highest on a particular dimension. Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
   References
   Amabile, T. M. (1982). Social psychology of creativity: A consensual assessment technique. Journal of Personality and Social
   Psychology, 43, 997–1013. doi:10.1037/0022-3514.43.5.997 10.1037/0022-3514.43.5.997
   Google Scholar WorldCat Crossref
   Amabile, T. M. (1983). The social psychology of creativity. New York, NY: Springer.
   Google Scholar Google Preview WorldCat COPAC
   Amabile, T. M. (1996). Creativity in context: Update to “The Social Psychology of Creativity.” Boulder, CO: Westview Press.
   Baas, M., De Dreu, C. W., & Nijstad, B. A. (2008). A meta-analysis of 25 years of mood-creativity research: Hedonic tone,
   activation, or regulatory focus? Psychological Bulletin, 134, 779–806. doi:10.1037/a0012815 10.1037/a0012815
   Crossref
   Baas, M., De Dreu, C. W., & Nijstad, B. A. (2011). When prevention promotes creativity: The role of mood, regulatory focus, and
   regulatory closure. Journal of Personality and Social Psychology, 100, 794–809. doi:10.1037/a0022981 10.1037/a0022981
   Google Scholar WorldCat Crossref
   Baer, M. (2012). Putting creativity to work: The implementation of creative ideas in organizations. Academy of Management
   Journal, 55, 1102–1119. doi:10.5465/amj.2009.0470 10.5465/amj.2009.0470
   Google Scholar WorldCat Crossref
   Basadur, M., Runco, M. A., & Vega, L. A. (2000). Understanding how creative thinking skills, attitudes and behaviors work together:
   A causal process model. Journal of Creative Behavior, 34, 77–100. doi:10.1002/j.2162-6057.2000.tb01203.x 10.1002/j.2162-
   6057.2000.tb01203.x
   Google Scholar WorldCat Crossref
   Bechtoldt, M. N., De Dreu, C. W., Nijstad, B. A., & Choi, H. (2010). Motivated information processing, social tuning, and group
   creativity. Journal of Personality and Social Psychology, 99, 622–637. doi:10.1037/a0019386 10.1037/a0019386
   Google Scholar WorldCat Crossref
   Berg, J. M. (2016). Balancing on the creative highwire: Forecasting the success of novel ideas in organizations. Administrative
   Science Quarterly, 61, 433–468. doi:10.1177/0001839216642211. 10.1177/0001839216642211
   Google Scholar WorldCat Crossref
   Bittner, J. V., Bruena, M., & Rietzschel, E. F. (2016). Cooperation goals, regulatory focus, and their combined eects on creativity.
   Thinking Skills and Creativity, 19, 260–268. doi:10.1016/j.tsc.2015.12.002 10.1016/j.tsc.2015.12.002
   Google Scholar WorldCat Crossref
   Blair, C. S., & Mumford, M. D. (2007). Errors in idea evaluation: Preference for the unoriginal? Journal of Creative Behavior, 41,
   197–222. doi:10.1002/j.2162–6057.2007.tb01288.x 10.1002/j.2162–6057.2007.tb01288.x
   Google Scholar WorldCat Crossref
   Byrne, C. L., Shipman, A. S., & Mumford, M. D. (2010). The eects of forecasting on creative problem-solving: An experimental
   study. Creativity Research Journal, 22, 119–138. doi:10.1080/10400419.2010.481482 10.1080/10400419.2010.481482
   Google Scholar WorldCat Crossref
   Chirumbolo, A., Livi, S., Mannetti, L., Pierro, A., & Kruglanski, A. W. (2004). Eects of need for closure on creativity in small group
   interactions. European Journal of Personality, 18, 265–278. doi:10.1002/per.518 10.1002/per.518
   Google Scholar WorldCat Crossref
   Criscuolo, P., Dahlander, L., Grohsjean, T., & Salter, A. (2017). Evaluating novelty: The role of panels in the selection of R&D
   projects. Academy of Management Journal, 60, 433–460. doi:10.5465/amj.2014.0861 10.5465/amj.2014.0861
   p. 195
   Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
   Google Scholar WorldCat Crossref
   Crowe, E., & Higgins, E. T. (1997). Regulatory focus and strategic inclinations: Promotion and prevention in decision-making.
   Organizational Behavior and Human Decision Processes, 69, 117–132. doi:10.1006/obhd.1996.2675 10.1006/obhd.1996.2675
   Google Scholar WorldCat Crossref
   Csikszentmihalyi, M. (1999). Implications of a systems perspective for the study of creativity. In R. J. Sternberg (Ed.), Handbook of
   creativity (pp. 313–335). New York, NY: Cambridge University Press.
   Google Scholar Google Preview WorldCat COPAC
   Dailey, L., & Mumford, M. D. (2006). Evaluative aspects of creative thought: Errors in appraising the implications of new ideas.
   Creativity Research Journal, 18, 367–384. doi:10.1207/s15326934crj1803_11 10.1207/s15326934crj1803_11
   Google Scholar WorldCat Crossref
   Davis, J. H. (1973). Group decision and social interaction: A theory of social decision schemes. Psychological Review, 80, 97–125.
   doi:10.1037/h0033951 10.1037/h0033951
   Google Scholar WorldCat Crossref
   De Buisonjé, D. R., Ritter, S. M., de Bruin, S., ter Horst, J. M., & Meeldijk, A. (2017). Facilitating creative idea selection: The
   combined eects of self-airmation, promotion focus and positive aect. Creativity Research Journal, 29, 174–181.
   doi:10.1080/10400419.2017.1303308 10.1080/10400419.2017.1303308
   Google Scholar WorldCat Crossref
   De Dreu, C. K. W. (2010). Human creativity: Reflections on the role of culture. Management and Organization Review, 6, 437–446.
   doi:10.1111/j.1740–8784.2010.00195.x 10.1111/j.1740–8784.2010.00195.x
   Google Scholar WorldCat Crossref
   De Dreu, C. K. W., Baas, M., & Nijstad, B. A. (2008). Hedonic tone and activation level in the mood-creativity link: Toward a dual
   pathway to creativity model. Journal of Personality and Social Psychology, 94, 739–756. doi:10.1037/0022-
   3514.94.5.739 10.1037/0022-3514.94.5.739
   Google Scholar WorldCat Crossref
   Diehl, M., & Stroebe, W. (1987). Productivity loss in brainstorming groups: Toward the solution of a riddle. Journal of Personality
   and Social Psychology, 53, 497–509. doi: 10.1037/0022-3514.53.3.497 10.1037/0022-3514.53.3.497
   Google Scholar WorldCat Crossref
   Farrell, M. P. (2001). Collaborative circles: Friendship dynamics and creative work. Chicago, IL: University of Chicago Press.
   Google Scholar Google Preview WorldCat COPAC
   Faure, C. (2004). Beyond brainstorming: Eects of dierent group procedures on selection of ideas and satisfaction with the
   process. Journal of Creative Behavior, 38, 13–34.
   Google Scholar WorldCat
   Förster, J., Friedman, R. S., & Liberman, N. (2004). Temporal construal eects on abstract and concrete thinking: Consequences
   for insight and creative cognition. Journal of Personality and Social Psychology, 87, 177–189. doi:10.1037/0022-
   3514.87.2.177 10.1037/0022-3514.87.2.177
   Google Scholar WorldCat Crossref
   Frederiksen, M. H., & Knudsen, M. P. (2017). From creative ideas to innovation performance: The role of assessment criteria.
   Creativity and Innovation Management, 26, 60–74. doi:10.1111/caim.12204 10.1111/caim.12204
   Google Scholar WorldCat Crossref
   Friedman, R. S., & Förster, J. (2001). The eects of promotion and prevention cues on creativity. Journal of Personality and Social
   Psychology, 81, 1001–1013. doi:10.1037/0022-3514.81.6.1001 10.1037/0022-3514.81.6.1001
   Google Scholar WorldCat Crossref
   Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
   Fürst, G., Ghisletta, P., & Lubart, T. (2016). Toward an integrative model of creativity and personality: Theoretical suggestions and
   preliminary empirical testing. Journal of Creative Behavior, 50, 87–108. doi:10.1002/jocb.71 10.1002/jocb.71
   Google Scholar WorldCat Crossref
   Girotra, K., Terwiesch, C., & Ulrich, K. T. (2010). Idea generation and the quality of the best idea. Management Science, 56, 591–
3. doi:10.1287/mnsc.1090.1144 10.1287/mnsc.1090.1144
   Google Scholar WorldCat Crossref
   Guerrero, R. (2008). The session that did not shake the world (the Linnean Society, July 1, 1858). International Microbiology, 11,
   209–212.
   Google Scholar WorldCat
   Hao, N., Ku, Y., Liu, M., Hu, Y., Bodner, M., Grabner, R. H., & Fink, A. (2016). Reflection enhances creativity: Beneficial eects of idea
   evaluation on idea generation. Brain and Cognition, 10, 330–337. doi:10.1016/j.bandc.2016.01.005 10.1016/j.bandc.2016.01.005
   Google Scholar WorldCat Crossref
   Hiskey, D. (2001). Today I found out, post-it notes were developed by accident. Retrieved from
   http://www.todayifoundout.com/index.php/2011/11/post-it-notes-were-invented-by-accident/
   WorldCat
   Harvey, S. (2013). A dierent perspective: The multiple eects of deep level diversity on group creativity. Journal of Experimental
   Social Psychology, 49, 822–832. doi:10.1016/j.jesp.2013.04.004 10.1016/j.jesp.2013.04.004
   Google Scholar WorldCat Crossref
   Harvey, S., & Kou, C.-Y. (2013). Collective engagement in creative tasks: The role of evaluation in the creative process in groups.
   Administrative Science Quarterly, 58, 346–386. doi:10.1177/0001839213498591 10.1177/0001839213498591
   Google Scholar WorldCat Crossref
   Herman, A., & Reiter-Palmon, R. (2011). The eect of regulatory focus on idea generation and idea evaluation. Psychology of
   Aesthetics, Creativity, and the Arts, 5, 13–20. doi:10.1037/a0018587 10.1037/a0018587
   Google Scholar WorldCat Crossref
   Higgins, E. T. (1997). Beyond pleasure and pain. American Psychologist, 52, 1280–1300. doi:10.1037/0003-
   066X.52.12.1280 10.1037/0003-066X.52.12.1280
   Google Scholar WorldCat Crossref
   Hocevar, D. (1979). A comparison of statistical infrequency and subjective judgment as criteria in the measurement of originality.
   Journal of Personality Assessment, 43, 297–299. doi:10.1207/s15327752jpa4303_13 10.1207/s15327752jpa4303_13
   Google Scholar WorldCat Crossref
   Jia, L., Hirt, E. R., & Karpen, S. C. (2009). Lessons from a faraway land: The eect of spatial distance on creative cognition. Journal
   of Experimental Social Psychology, 45, 1127–1131. doi:10.1016/j.jesp.2009.05.015 10.1016/j.jesp.2009.05.015
   Google Scholar WorldCat Crossref
   Kaminski, M. (2007). The secret history of Star Wars. Kingston, ON, Canada: Legacy Books Press.
   Google Scholar Google Preview WorldCat COPAC
   Kaufman, J. C., Baer, J., Cole, J. C., & Sexton, J. D. (2008). A comparison of expert and nonexpert raters using the consensual
   assessment technique. Creativity Research Journal, 20, 171–178. doi:10.1080/10400410802059929 10.1080/10400410802059929
   Google Scholar WorldCat Crossref
   Kaufman, J. C., Baer, J., Cropley, D. H., Reiter-Palmon, R., & Sinnett, S. (2013). Furious activity vs. understanding: How much
   expertise is needed to evaluate creative work? Psychology of Aesthetics, Creativity, and the Arts, 7, 332–340.
   doi:10.1037/a0034809 10.1037/a0034809
   Google Scholar WorldCat Crossref
   p. 196
   Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
   Kerr, N. L., MacCoun, R. J., & Kramer, G. P. (1996). Bias in judgment: Comparing individuals and groups. Psychological Review,
   103, 687–719. doi:10.1037/0033-295X.103.4.687 10.1037/0033-295X.103.4.687
   Google Scholar WorldCat Crossref
   Kohn, N. W., Paulus, P. B., & Choi, Y. (2011). Building on the ideas of others: An examination of the idea combination process.
   Journal of Experimental Social Psychology, 47, 554–561. doi:10.1016/j.jesp.2011.01.004 10.1016/j.jesp.2011.01.004
   Google Scholar WorldCat Crossref
   Larey, T. S. (1994). Convergent and divergent thinking, group composition, and creativity in brainstorming groups (Unpublished
   doctoral dissertation). University of Texas at Arlington.
   Larey, T. S., & Paulus, P. B. (1999). Group preference and convergent tendencies in small groups: A content analysis of group
   brainstorming performance. Creativity Research Journal, 12, 175–184.
   doi:10.1207/s15326934crj1203_2 10.1207/s15326934crj1203_2
   Google Scholar WorldCat Crossref
   Laughlin, P. R., & Ellis, A. L. (1986). Demonstrability and social combination processes on mathematical intellective tasks.
   Journal of Experimental Social Psychology, 22, 177–189. doi:10.1016/0022-1031(86)90022-3 10.1016/0022-1031(86)90022-3
   Google Scholar WorldCat Crossref
   Levine, J. M., Higgins, E. T., & Choi, H. (2000). Development of strategic norms in groups. Organizational Behavior and Human
   Decision Processes, 82, 88–101. doi:10.1006/obhd.2000.2889 10.1006/obhd.2000.2889
   Google Scholar WorldCat Crossref
   Levy, S. (2017). Google Glass 2.0 is a startling second act. Wired Magazine, July 18, 2017. Retrieved from
   https://www.wired.com/story/google-glass-2-is-here/
   Google Scholar WorldCat
   Litchfield, R. C., Gilson, L. L., & Gilson, P. W. (2015). Defining creative ideas: Toward a more nuanced approach. Group and
   Organization Management, 40, 238–265. doi:10.1177/1059601115574945 10.1177/1059601115574945
   Google Scholar WorldCat Crossref
   Lonergan, D. C., Scott, G. M., & Mumford, M. D. (2004). Evaluative aspects of creative thought: Eects of appraisal and revision
   standards. Creativity Research Journal, 16, 231–246. doi:10.1207/s15326934crj1602&3_7 10.1207/s15326934crj1602&3_7
   Google Scholar WorldCat Crossref
   McMahon, K., Ruggeri, A., Kämmer, J. E., & Katsikopoulos, K. V. (2016). Beyond idea generation: The power of groups in
   developing ideas. Creativity Research Journal, 28, 247–257.
   doi:10.1080/10400419.2016.1195637 10.1080/10400419.2016.1195637
   Google Scholar WorldCat Crossref
   Miron-Spektor, E., & Beenen, G. (2015). Motivating creativity: The eects of sequential and simultaneous learning and
   performance achievement goals on product novelty and usefulness. Organizational Behavior and Human Decision Processes,
   127, 53–65. doi:10.1016/j.obhdp.2015.01.001 10.1016/j.obhdp.2015.01.001
   Google Scholar WorldCat Crossref
   Miron-Spektor, E., Gino, F., & Argote, L. (2011). Paradoxical frames and creative sparks: Enhancing individual creativity through
   conflict and integration. Organizational Behavior and Human Decision Processes, 116, 229–240.
   doi:10.1016/j.obhdp.2011.03.006 10.1016/j.obhdp.2011.03.006
   Google Scholar WorldCat Crossref
   Moreau, C. P., Lehmann, D. R., & Markman, A. B. (2001). Entrenched knowledge structures and consumer responses to new
   products. Journal of Marketing Research, 38, 14–29. doi:10.1509/jmkr.38.1.14.18836 10.1509/jmkr.38.1.14.18836
   Google Scholar WorldCat Crossref
   Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
   Mueller, J. S., Melwani, S., & Goncalo, J. A. (2012). The bias against creativity: Why people desire but reject creative ideas.
   Psychological Science, 23, 13–17. doi:10.1177/0956797611421018 10.1177/0956797611421018
   Google Scholar WorldCat Crossref
   Mueller, J. S., Wakslak, C. J., & Krishnan, V. (2014). Construing creativity: The how and why of recognizing creative ideas. Journal
   of Experimental Social Psychology, 51, 81–87. doi:10.1016/j.jesp.2013.11.007 10.1016/j.jesp.2013.11.007
   Google Scholar WorldCat Crossref
   Mumford, M. D., Lonergan, D. C., & Scott, G. (2002). Evaluating creative ideas: Processes, standards, and context. Inquiry: Critical
   Thinking Across the Disciplines, 22, 21–30. doi:10.5840/inquiryctnews20022213 10.5840/inquiryctnews20022213
   Google Scholar WorldCat Crossref
   Naughton, J. (2017, July 23). The rebirth of Google Glass shows the merit of failure. The Guardian. Retrieved from
   https://www.theguardian.com/commentisfree/2017/jul/23/the-return-of-google-glass-surprising-merit-in-failure-enterpriseedition
   Google Scholar WorldCat
   Nijstad, B. A., De Dreu, C. W., Rietzschel, E. F., & Baas, M. (2010). The dual pathway to creativity model: Creative ideation as a
   function of flexibility and persistence. European Review of Social Psychology, 21, 34–77.
   doi:10.1080/10463281003765323 10.1080/10463281003765323
   Google Scholar WorldCat Crossref
   Nijstad, B. A., & Stroebe, W. (2006). How the group aects the mind: A cognitive model of idea generation in groups. Personality
   and Social Psychology Review, 10, 186–213. doi:10.1207/s15327957pspr1003_1 10.1207/s15327957pspr1003_1
   Google Scholar WorldCat Crossref
   Onarheim, B., & Christensen, B. T. (2012). Distributed idea screening in stage-gate development processes. Journal of
   Engineering Design, 23, 660–673. doi: 10.1080/09544828.2011.649426 10.1080/09544828.2011.649426
   Google Scholar WorldCat Crossref
   Osborn, A. F. (1957). Applied imagination (2nd rev. ed.). Oxford, UK: Scribner.
   Google Scholar Google Preview WorldCat COPAC
   Perry-Smith, J. E., & Co, R. W. (2011). In the mood for entrepreneurial creativity? How optimal group aect diers for generating
   and selecting ideas for new ventures. Strategic Entrepreneurship Journal (Vol. 5, pp. 247–268). doi:
   10.1002/sej.116 10.1002/sej.116
   Google Scholar WorldCat Crossref
   Petty, R. E., Fleming, M. A., & Fabrigar, L. R. (1999). The review process at PSPB: Correlates of interreviewer agreement and
   manuscript acceptance. Personality and Social Psychology Bulletin, 25, 188–203.
   doi:10.1177/0146167299025002005 10.1177/0146167299025002005
   Google Scholar WorldCat Crossref
   Putman, V. L., & Paulus, P. B. (2009). Brainstorming, brainstorming rules and decision making. Journal of Creative Behavior, 43,
   23–39. doi:10.1002/j.2162-6057.2009.tb01304.x 10.1002/j.2162-6057.2009.tb01304.x
   Google Scholar WorldCat Crossref
   Reiter-Palmon, R., & Arreola, N. J. (2015). Does generating multiple ideas lead to increased creativity? A comparison of generating
   one idea vs many. Creativity Research Journal, 27, 369–374.
   doi:10.1080/10400419.2015.1087274 10.1080/10400419.2015.1087274
   Google Scholar WorldCat Crossref
   Rietzschel, E. F. (2011). Collective regulatory focus predicts specific aspects of team innovation. Group Processes and Intergroup
   Relations, 14, 337–345. doi:10.1177/1368430210392396 10.1177/1368430210392396
   Google Scholar WorldCat Crossref
   Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
   Rietzschel, E. F., Nijstad, B. A., & Stroebe, W. (2006). Productivity is not enough: A comparison of interactive and nominal
   brainstorming groups on idea generation and selection. Journal of Experimental Social Psychology, 42, 244–251.
   doi:10.1016/j.jesp.2005.04.005 10.1016/j.jesp.2005.04.005
   Google Scholar WorldCat Crossref
   Rietzschel, E. F., Nijstad, B. A., & Stroebe, W. (2010). The selection of creative ideas aer individual idea generation: Choosing
   between creativity and impact. British Journal of Psychology, 101, 47–68.
   doi:10.1348/000712609X414204 10.1348/000712609X414204
   Google Scholar WorldCat Crossref
   Rietzschel, E. F., Nijstad, B. A., & Stroebe, W. (2014). Eects of problem scope and creativity instructions on idea generation
   and selection. Creativity Research Journal, 26, 185–191. doi:10.1080/10400419.2014.901084 10.1080/10400419.2014.901084
   Crossref
   Ritter, S. M., van Baaren, R. B., & Dijksterhuis, A. (2012). Creativity: The role of unconscious processes in idea generation and idea
   selection. Thinking Skills and Creativity, 7, 21–27. doi:10.1016/j.tsc.2011.12.002 10.1016/j.tsc.2011.12.002
   Google Scholar WorldCat Crossref
   Runco, M. A., & Charles, R. E. (1993). Judgments of originality and appropriateness as predictors of creativity. Personality and
   Individual Dierences, 15, 537–546.
   Google Scholar WorldCat
   Runco, M. A., & Dow, G. T. (2004). Assessing the accuracy of judgments of originality on three divergent thinking tests. Korean
   Journal of Thinking and Problem Solving, 14, 5–14.
   Google Scholar WorldCat
   Runco, M. A., Illies, J. J., & Eisenman, R. (2005). Creativity, Originality, and Appropriateness: What do Explicit Instructions Tell Us
   about Their Relationships? The Journal of Creative Behavior, 39, 137–148. doi:10.1002/j.2162-
   6057.2005.tb01255.x 10.1002/j.2162-6057.2005.tb01255.x
   Google Scholar WorldCat Crossref
   Runco, M. A., & Smith, W. R. (1992). Interpersonal and intrapersonal evaluations of creative ideas. Personality and Individual
   Dierences, 13, 295–302. doi:10.1016/0191-8869(92)90105-X 10.1016/0191-8869(92)90105-X
   Google Scholar WorldCat Crossref
   Schimmel, K., & Förster, J. (2008). How temporal distance changes novicesʼ attitudes towards unconventional arts. Psychology of
   Aesthetics, Creativity, and the Arts, 2, 53–60. doi:10.1037/1931-3896.2.1.53 10.1037/1931-3896.2.1.53
   Google Scholar WorldCat Crossref
   Silvia, P. J. (2008). Discernment and creativity: How well can people identify their most creative ideas? Psychology of Aesthetics,
   Creativity, and the Arts, 2, 139–146. doi:10.1037/1931-3896.2.3.139 10.1037/1931-3896.2.3.139
   Google Scholar WorldCat Crossref
   Simonton, D. K. (2018). Defining creativity: Donʼt we also need to define what is not creative? Journal of Creative Behavior, 52, 80–
4. doi:10.1002/jocb.137 10.1002/jocb.137
   Crossref
   Simonton, D. K. (2016). Giving credit where creditʼs due: Why itʼs so hard to do in psychological science. Perspectives on
   Psychological Science, 11, 888–892. doi:10.1177/1745691616660155 10.1177/1745691616660155
   Google Scholar WorldCat Crossref
   Stam, D., De Vet, A., Barkema, H. G., & De Dreu, C. K. W. (2013). Suspending group debate and developing concepts. Journal of
   Product Innovation Management, 30, 48–61.
   Google Scholar WorldCat
   p. 197
   Downloaded from https://academic.oup.com/edited-volume/28144/chapter/212910364 by Rijksuniversiteit Groningen user on 09 June 2023
   Stasser, G. (1999). A primer of social decision scheme theory: Models of group influence, competitive model-testing, and
   prospective modeling. Organizational Behavior and Human Decision Processes, 80, 3–20.
   doi:10.1006/obhd.1999.2851 10.1006/obhd.1999.2851
   Google Scholar WorldCat Crossref
   Sternberg, R. J., & Davidson, J. E. (1995). The nature of insight. Cambridge, MA: MIT Press.
   Google Scholar Google Preview WorldCat COPAC
   Sternberg, R. J., & Lubart, T. I. (1999). The concept of creativity: Prospects and paradigms. In R. J. Sternberg (Ed.), Handbook of
   creativity (pp. 3–15). New York, NY: Cambridge University Press.
   Google Scholar Google Preview WorldCat COPAC
   Stroebe, W., Nijstad, B. A., & Rietzschel, E. F. (2010). Beyond productivity loss in brainstorming groups: The evolution of a
   question. In M. P. Zanna, J. M. Olson (Eds.), Advances in experimental social psychology (Vol. 43, pp. 157–203). San Diego, CA:
   Academic Press. doi:10.1016/S0065-2601(10)43004-X 10.1016/S0065-2601(10)43004-X
   Google Scholar Google Preview WorldCat COPAC Crossref
   Toh, C. A., & Miller, S. R. (2015). How engineering teams select design concepts: A view through the lens of creativity. Design
   Studies, 38, 111–138.
   Google Scholar WorldCat
   Toh, C. A., & Miller, S. R. (2016a). Creativity in design teams: The influence of personality traits and risk attitudes on creative
   concept selection. Research in Engineering Design, 27, 73–89.
   Google Scholar WorldCat
   Toh, C. A., & Miller, S. R. (2016b). Choosing creativity: The role of individual risk and ambiguity aversion on creative concept
   selection in engineering design. Research in Engineering Design, 27, 195–219.
   Google Scholar WorldCat
   Trope, Y., & Liberman, N. (2010). Construal-level theory of psychological distance. Psychological Review, 117, 440–463.
   doi:10.1037/a0018963 10.1037/a0018963
   Google Scholar WorldCat Crossref
   Wallas, G. (1926). The art of thought. London, UK: J. Cape.
   Google Scholar Google Preview WorldCat COPAC
   Ward, T. B. (1994). Structured imagination: The role of category structure in exemplar generation. Cognitive Psychology, 27, 1–40.
   doi:10.1006/cogp.1994.1010 10.1006/cogp.1994.1010
   Google Scholar WorldCat Crossref
   Zacher, H., Robinson, A. J., & Rosing, K. (2016). Ambidextrous leadership and employeesʼ self-reported innovative performance:
   The role of exploration and exploitation behaviors. Journal of Creative Behavior, 50, 24–46. doi:10.1002/jocb.66 10.1002/jocb.66
   Google Scholar WorldCat Crossref
   Zhou, J., Wang, X. M., Song, L. J., & Wu, J. (2017). Is it new? Personal and contextual influences on perceptions of novelty and
   creativity. Journal of Applied Psychology, 102, 180–202. doi:10.1037/apl0000166 10.1037/apl0000166
   Google Scholar WorldCat Crossref
   Zhu, Y., Ritter, S. M., Müller, B. C. N., & Dijksterhuis, A. (2017). Creativity: Intuitive processing outperforms deliberative processing
   in creative idea selection. Journal of Experimental Social Psychology, 73, 180–188.
   doi:10.1016/j.jesp.2017.06.009 10.1016/j.jesp.2017.06.009
   Google Scholar WorldCat Crossref