A Hierarchical Approach to Judgment Aggregation with Abstentions

Judgment aggregation deals with the problem of how collective judgments on logically connected propositions can be formed based on individual judgments on the same propositions. The existing literature on judgment aggregation mainly focuses on the anonymity condition requiring that individual judgments be treated equally. However, in many real‐world situations, a group making collective judgments may assign individual members or subgroups different priorities to determine the collective judgment. Based on this consideration, this article relaxes the anonymity condition by giving a hierarchy over individuals so as to investigate how the judgment from each individual affects the group judgment in such a hierarchical environment. Moreover, we assume that an individual can abstain from voting on a proposition and the collective judgment on a proposition can be undetermined, which means that we do not require completeness at both individual and collective levels. In this new setting, we first identify an impossibility result and explore a set of plausible conditions in terms of abstentions. Secondly, we develop an aggregation rule based on the hierarchy of individuals and show that the aggregation rule satisfies those plausible conditions. The computational complexity of this rule is also investigated. Finally, we show that the proposed rule is (weakly) oligarchic over a subset of agenda. This is by no means a negative result. In fact, our result reveals that with abstentions, oligarchic aggregation is not necessary to be a single‐level determination but can be a multiple‐level collective decision making, which partially explains its ubiquity in the real world.     

Natural Deduction for Modal Logic of Judgment Aggregation

We can formalize judgments as (consistent sets of) logical formulas. Judgment aggregation deals with judgments of several agents, which need to be aggregated to a collective judgment. There are several logical formalizations of judgment aggregation. This paper focuses on a modal formalization which nicely expresses classical properties of judgment aggregation rules and famous results of social choice theory, like Arrow’s impossibility theorem. A natural deduction system for modal logic of judgment aggregation is presented in this paper. The system is sound and complete. As an example of derivation, a formal proof of Arrow’s impossibility theorem is given.     

On judgment aggregation in abstract argumentation

Judgment aggregation is a field in which individuals are required to vote for or against a certain decision (the conclusion) while providing reasons for their choice. The reasons and the conclusion are logically connected propositions. The problem is how a collective judgment on logically interconnected propositions can be defined from individual judgments on the same propositions. It turns out that, despite the fact that the individuals are logically consistent, the aggregation of their judgments may lead to an inconsistent group outcome, where the reasons do not support the conclusion. However, in this paper we claim that collective irrationality should not be the only worry of judgment aggregation. For example, judgment aggregation would not reject a consistent combination of reasons and conclusion that no member voted for. In our view this may not be a desirable solution. This motivates our research about when a social outcome is ‘compatible’ with the individuals’ judgments. The key notion that we want to capture is that any individual member has to be able to defend the collective decision. This is guaranteed when the group outcome is compatible with its members views. Judgment aggregation problems are usually studied using classical propositional logic. However, for our analysis we use an argumentation approach to judgment aggregation problems. Indeed the question of how individual evaluations can be combined into a collective one can also be addressed in abstract argumentation. We introduce three aggregation operators that satisfy the condition above, and we offer two definitions of compatibility. Not only does our proposal satisfy a good number of standard judgment aggregation postulates, but it also avoids the problem of individual members of a group having to become committed to a group judgment that is in conflict with their own individual positions.     

Boosting metacomprehension accuracy in computer-supported learning: The role of judgment task and judgment scope

Although learners' judgments of their own learning are crucial for self-regulated study, judgment accuracy tends to be low. To increase accuracy, we had participants make combined judgments. In Experiment 1, 247 participants studied a ten-chapter expository text. In the simple judgments group, participants after each chapter rated the likelihood of answering correctly a knowledge question on that chapter (judgment of learning; JOL). In the combined judgments group, participants rated text difficulty before making a JOL. No accuracy differences emerged between groups, but a comparison of early-chapter and late-chapter judgment magnitudes showed that the judgment manipulation had induced cognitive processing differences. In Experiment 2, we therefore manipulated judgment scope. Rather than predicting answers correct for an entire chapter, another 256 participants rated after each chapter the likelihood of answering correctly a question on a specific concept from that chapter. Both judgment accuracy and knowledge test scores were higher in the combined judgments group. Moreover, while judgment accuracy dropped to an insignificant level between early and late chapters in the simple judgments group, accuracy remained constant with combined judgments. We discuss implications for research into metacomprehension processes in computer-supported learning and for adaptive learner support based on judgment prompts.     

On the order of the preference intensities in fuzzy AHP

We show that a recently discovered fundamental problem with the Analytic Hierarchy Process (AHP) concerning the meaning of the resultant preference intensities is also evident for the fuzzy AHP. We prove that if there is a judgmental inconsistency in the fuzzy pair-wise comparisons, it is impossible to ensure the preservation of the order regarding to preference intensities in the resultant priority vector. Further, it is shown with an example from the published literature that the order of the preference intensities may not be preserved even there is no inconsistency in the judgment set, albeit it is possible to comply with this order via using fuzzy preference programming (FPP) methodology. Finally, it is proved that if the interval judgments regarding to the decompositions of original judgments to - level sets are consistent, FPP guarantees the preservation of the order of the preference intensities at those levels.     

On the logic of preference and judgment aggregation

Agents that must reach agreements with other agents need to reason about how their preferences, judgments, and beliefs might be aggregated with those of others by the social choice mechanisms that govern their interactions. The emerging field of judgment aggregation studies aggregation from a logical perspective, and considers how multiple sets of logical formulae can be aggregated to a single consistent set. As a special case, judgment aggregation can be seen to subsume classical preference aggregation. We present a modal logic that is intended to support reasoning about judgment aggregation scenarios (and hence, as a special case, about preference aggregation): the logical language is interpreted directly in judgment aggregation rules. We present a sound and complete axiomatisation. We show that the logic can express aggregation rules such as majority voting; rule properties such as independence; and results such as the discursive paradox, Arrow’s theorem and Condorcet’s paradox—which are derivable as formal theorems of the logic. The logic is parameterised in such a way that it can be used as a general framework for comparing the logical properties of different types of aggregation—including classical preference aggregation. As a case study we present a logical study of, including a formal proof of, the neutrality lemma, the main ingredient in a well-known proof of Arrow’s theorem.     

Images in computer-supported learning: Increasing their benefits for metacomprehension through judgments of learning

Images are widely used in computer-based learning although they might bias learners’ judgments on how well they have mastered the material, which might reduce the effectiveness of metacognitive learning control. As that bias seems to result from primarily theory-based processing, we used two-step judgments of learning to induce more experience-based processes that we hypothesized to benefit metacomprehension accuracy. In Experiment 1, 381 participants studied a ten-section text on team building that was either accompanied by conceptual images, decorative images, or no images. Half of the participants made simple judgments by rating after each section the likelihood of correctly answering a knowledge question on that section (judgment of learning; JOL) The other half made combined judgments by rating text difficulty before making a JOL. As postulated, combined JOL benefitted accuracy and knowledge test scores; both were highest in the conceptual images group. In Experiment 2, to further increase accuracy, we manipulated judgment scope. Rather than predicting answers correct for an entire chapter, another 310 participants had to predict answers correct for a specific concept from a chapter. Again, accuracy and test scores were highest in the conceptual images group. Contrary to expectations, however, JOL accuracy did not benefit from term-specific judgments. We discuss implications for research into metacomprehension processes in computer-supported learning and for adaptive learner support based on judgment prompts.     

Effect of verbal comprehension skill and self-reported features on reliability of crowdsourced relevance judgments

Test collection is extensively used to evaluate information retrieval systems in laboratory-based evaluation experimentation. In a classic setting of a test collection, human assessors involve relevance judgments, which are both a costly and time-consuming task, and scales poorly. Researchers are still being challenged in performing reliable and low-cost evaluation of retrieval systems. Crowdsourcing as a novel method provides a cost-effective and quick solution for creating relevance judgments. However, crowdsourcing comes with the risk of a heterogeneous mass of potential workers who create the relevance judgments with varied levels of accuracy. It is, therefore, essential to understand the factors that affect the reliability of crowdsourced judgments. In this article, we measured various cognitive characteristics of workers, and explored the effects of these characteristics on judgment reliability, in comparison with a human gold standard. We discovered a significant correlation between judgment reliability and the level of verbal comprehension skill. This association conveys an idea for improving the reliability of judgments by discriminating workers into various groups according to their cognitive abilities and to filter out (or to include) certain group(s) of workers. Aside from that, we also discovered a significant association between reliability of judgments and self-reported difficulty of judgment as well as confidence in the task.     

复杂网络上的群体决策

现状和最新进展.首先介绍了观点动力学研究中的几种基本模型,即Ising模型、投票者模型、多数决定模型和有界自信模型等.其次以这些模型为基础,讨论了小世界、无标度等复杂网络上观点动力学的研究结果,然后指出了观点动力学与语言游戏、一致性和耦合振子同步问题的联系,接着给出了笔者在观点动力学方面所做的一些相关工作,最后指出了复杂网络上群体决策的未来发展方向和一些可能的应用前景.     

Estimating average precision when judgments are incomplete

We consider the problem of evaluating retrieval systems with incomplete relevance judgments. Recently, Buckley and Voorhees showed that standard measures of retrieval performance are not robust to incomplete judgments, and they proposed a new measure, bpref, that is much more robust to incomplete judgments. Although bpref is highly correlated with average precision when the judgments are effectively complete, the value of bpref deviates from average precision and from its own value as the judgment set degrades, especially at very low levels of assessment. In this work, we propose three new evaluation measures induced AP, subcollection AP, and inferred AP that are equivalent to average precision when the relevance judgments are complete and that are statistical estimates of average precision when relevance judgments are a random subset of complete judgments. We consider natural scenarios which yield highly incomplete judgments such as random judgment sets or very shallow depth pools. We compare and contrast the robustness of the three measures proposed in this work with bpref for both of these scenarios. Through the use of TREC data, we demonstrate that these measures are more robust to incomplete relevance judgments than bpref, both in terms of how well the measures estimate average precision (as measured with complete relevance judgments) and how well they estimate themselves (as measured with complete relevance judgments). Finally, since inferred AP is the most accurate approximation to average precision and the most robust measure in the presence of incomplete judgments, we provide a detailed analysis of this measure, both in terms of its behavior in theory and its implementation in practice. We gratefully acknowledge the support provided by NSF grants CCF-0418390 and IIS-0534482.     

一种基于群组判断离差的偏好集结方法

针对群决策偏好集结中违反Pareto 最优性的情况, 设计一种基于群组判断几何离差的同质性集结方法. 该方法在集结前进行几何离差测试, 以确定个体决策信息的离差水平. 离差较小时, 可基于几何平均集结; 对于离差较大且修正复杂度较高的决策信息, 采用主成分分析(PCA) 从高维决策信息中提取大多数相关信息, 在不依赖主观分析的情况下进行加权集结. 仿真实验表明, 所提出的方法能够在不违背Pareto 最优性的基础上集结离差较大的群决策信息.

     

Interval Type-2 fuzzy voter design for fault tolerant systems

A voting scheme constitutes an essential component of many fault tolerant systems. Two types of voters are commonly used in applications of real-valued systems: the inexact majority and the amalgamating voters. The inexact majority voter effectively isolates erroneous modules and is capable of reporting benign outputs when a significant disagreement is detected. However, an application specific voter threshold must be provided. On the other hand, amalgamating voter, such as the weighted average voter, reduces the influence of faulty modules by averaging the input values together. Unlike the majority voters, amalgamating voters are not capable of producing benign outputs. In the past, a Type-1 (T1) fuzzy voting scheme was introduced, allowing for both smooth amalgamation of voter inputs and effective signalization of benign outputs. The presented paper proposes an extension to the fuzzy voting scheme via incorporating Interval Type-2 (IT2) fuzzy logic. The IT2 fuzzy logic allows for an improved handling of uncertain assumptions about the distributions of noisy and erroneous inputs which are essential for correct design of the fuzzy voting scheme. The proposed voter design features robust performance when the uncertainty assumptions dynamically change over time. The IT2 fuzzy voter architecture was compared against the average voter, inexact majority voter, and the T1 fuzzy voter using a refined experimental harness. The reported results demonstrate improved availability, safety and reliability of the presented IT2 fuzzy voting scheme.     

Measuring the fit between human judgments and automated alerting algorithms: a study of collision detection

Methodologies for assessing human judgment in complex domains are important for the design of both displays that inform judgment and automated systems that suggest judgments. This paper uses the n-system lens model to evaluate the impact of displays on human judgment and to explicitly assess the similarity between human judgments and a set of potential judgment algorithms for use in automated systems. First, the need for and concepts underlying judgment analysis are outlined. Then the n-system lens model and its parameters are formally described. This model is then used to examine a previously conducted study of aircraft collision detection that had been analyzed using standard analysis of variance methods. Our analysis found the same main effects as did the earlier analysis. However, n-system lens model analysis was able to provide greater insight into the information relied upon for judgments and the impact of displays on judgment. Additionally, the analysis was able to identify attributes of human judgments that were--and were not--similar to judgments produced by automated systems. Potential applications of this research include automated aid design and operator training.     

Evaluating multifactor risks under conceptual uncertainty

A tool is proposed to evaluate multifactor risks during the operation of an innovation system of technological forecast. A modified BOCR method of analytic hierarchy process (AHP) is developed. It allows: integrating the situation and force majeur risk evaluation into an overall structure of decision making with the help of AHP along with evaluating benefits, costs, and opportunities for each alternative; processing expert judgments in the form of fuzzy preference relations; taking into account a time parameter, when decision factors and alternatives may be corrected or fundamentally changed during some period. Indices of the risk of subjective judgment (information risk) evaluation are developed for given point, interval, and fuzzy expert judgments and probability distribution of expert judgments. Translated from Kibernetika i Sistemnyi Analiz, No. 2, pp. 72–82, March–April 2009.     

Developing global managers’ competencies using the fuzzy DEMATEL method

Modern global managers are required to possess a set of competencies or multiple intelligences in order to meet pressing business challenges. Hence, expanding global managers’ competencies is becoming an important issue. Many scholars and specialists have proposed various competency models containing a list of required competencies. But it is hard for someone to master a broad set of competencies at the same time. Here arises an imperative issue on how to enrich global managers’ competencies by way of segmenting a set of competencies into some portions in order to facilitate competency development with a stepwise mode. To solve this issue involving the vagueness of human judgments, we have proposed an effective method combining fuzzy logic and Decision Making Trial and Evaluation Laboratory (DEMATEL) to segment required competencies for better promoting the competency development of global managers. Additionally, an empirical study is presented to illustrate the application of the proposed method.     

面向机器道德判断任务的细粒度中文道德语义知识库构建

道德智慧——一种做出道德判断的能力,是人类智力的一个独特元素。让机器学习人类的道德判断能力,做出符合社会道德观念的判断,是机器伦理领域的重要研究问题。目前,道德判断相关研究多为基于英文背景的简单粗粒度判断,缺少适用于中国社会的、细粒度分类的道德知识数据资源。基于此,该文提出面向机器道德判断的细粒度中文道德语义知识库构建任务,设计适用于中国社会的道德行为归类体系、道德框架表示体系和道德强度衡量体系三个维度的理论体系,构建了包含15 371词的中文道德语义知识库。     

Multimodal semi-automated affect detection from conversational cues,gross body language,and facial features

We developed and evaluated a multimodal affect detector that combines conversational cues, gross body language, and facial features. The multimodal affect detector uses feature-level fusion to combine the sensory channels and linear discriminant analyses to discriminate between naturally occurring experiences of boredom, engagement/flow, confusion, frustration, delight, and neutral. Training and validation data for the affect detector were collected in a study where 28 learners completed a 32- min. tutorial session with AutoTutor, an intelligent tutoring system with conversational dialogue. Classification results supported a channel × judgment type interaction, where the face was the most diagnostic channel for spontaneous affect judgments (i.e., at any time in the tutorial session), while conversational cues were superior for fixed judgments (i.e., every 20 s in the session). The analyses also indicated that the accuracy of the multichannel model (face, dialogue, and posture) was statistically higher than the best single-channel model for the fixed but not spontaneous affect expressions. However, multichannel models reduced the discrepancy (i.e., variance in the precision of the different emotions) of the discriminant models for both judgment types. The results also indicated that the combination of channels yielded superadditive effects for some affective states, but additive, redundant, and inhibitory effects for others. We explore the structure of the multimodal linear discriminant models and discuss the implications of some of our major findings.     

Learning to predict opinion share and detect anti-majority opinionists in social networks

We address the problem of detecting anti-majority opinionists using the value-weighted mixture voter (VwMV) model. This problem is motivated by the fact that 1) each opinion has its own value and an opinion with a higher value propagates more easily/rapidly and 2) there are always people who have a tendency to disagree with any opinion expressed by the majority. We extend the basic voter model to include these two factors with the value of each opinion and the anti-majoritarian tendency of each node as new parameters, and learn these parameters from a sequence of observed opinion data over a social network. We experimentally show that it is possible to learn the opinion values correctly using a short observed opinion propagation data and to predict the opinion share in the near future correctly even in the presence of anti-majoritarians, and also show that it is possible to learn the anti-majoritarian tendency of each node if longer observation data is available. Indeed, the learned model can predict the future opinion share much more accurately than a simple polynomial extrapolation can do. Ignoring these two factors substantially degrade the performance of share prediction. We also show theoretically that, in a situation where the local opinion share can be approximated by the average opinion share, 1) when there are no anti-majoritarians, the opinion with the highest value eventually takes over, but 2) when there are a certain fraction of anti-majoritarians, it is not necessarily the case that the opinion with the highest value prevails and wins, and further, 3) in both cases, when the opinion values are uniform, the opinion share prediction problem becomes ill-defined and any opinion can win. The simulation results support that this holds for typical real world social networks. These theoretical results help understand the long term behavior of opinion propagation.     

Exploring the effects of agent trust and benevolence in a simulated organizational task

Executives argue intuitively that trust is critical to effective organizational performance. Although articulated as a cognitive/ affective property ofindividuals, the collective effect of events influencing(and being influenced by) trust judgments must certainly impact organizational behavior. To begin to explore this, we conducted a simulation study of trust and organizational performance. Specifically, we defined a set ofcomputational agents, each with a trust function capable of evaluating the quality ofadvice from the other agents, and rendering judgments on the trustworthiness of the communicating agent. As agent judgments impact subsequent choices to accept or to generate communications, organizational performance is influenced. We manipulated two agent properties(trustworthiness, benevolence), two organizational variables (group size, group homogeneity/liar-to-honest ratio), and one environmental variable (stable, unstable). Results indicate that in homogeneous groups, honest groups did better than groups of liars, but under environmental instability, benevolent groups did worse. Under all conditions for heterogeneous groups, it only took one to three liars to degrade organizational performance.     

Multiple attribute group decision-making method with single-valued neutrosophic interval number information

To describe decision makers’ judgments for an evaluated object more comprehensively in the decision-making process, this paper firstly defines the concept of a single-valued neutrosophic interval number (SVNIN), which is composed of both its interval number (interval judgment) and its single-valued neutrosophic number (SVNN) (confident degree). Then we present the basic operational laws of SVNINs and the score function of SVNIN based on the attitude index and confident level of decision makers to compare SVNINs. Next, we propose a single-valued neutrosophic interval number weighted arithmetic averaging (SVNINWAA) operator and a single-valued neutrosophic interval number weighted geometric averaging (SVNINWGA) operator to aggregate SVNIN information. Further, a multiple attribute group decision-making (MDGDM) method using the proposed SVNINWAA or SVNINWGA operator is established under an SVNIN environment. Finally, an illustrative example regarding the selection problem of global suppliers is presented to show the application and effectiveness of the developed method.