start-ver=1.4 cd-journal=joma no-vol=11 cd-vols= no-issue= article-no= start-page=39466 end-page=39483 dt-received= dt-revised= dt-accepted= dt-pub-year=2023 dt-pub=20230413 dt-online= en-article= kn-article= en-subject= kn-subject= en-title= kn-title=Artificial Neural Network Based Audio Reinforcement for Computer Assisted Rote Learning en-subtitle= kn-subtitle= en-abstract= kn-abstract=The dual-channel assumption of the cognitive theory of multimedia learning suggests that importing a large amount of information through a single (visual or audio) channel overloads that channel, causing partial loss of information, while importing it simultaneously through multiple channels relieves the burden on them and leads to the registration of a larger amount of information. In light of such knowledge, this study investigates the possibility of reinforcing visual stimuli with audio for supporting e-learners in memorization tasks. Specifically, we consider three kinds of learning material and two kinds of audio stimuli and partially reinforce each kind of material with each kind of stimuli in an arbitrary way. In a series of experiments, we determine the particular type of audio, which offers the highest improvement for each kind of material. Our work stands out as being the first study investigating the differences in memory performance in relation to different combinations of learning content and stimulus. Our key findings from the experiments are: (i) E-learning is more effective in refreshing memory rather than studying from scratch, (ii) Non-informative audio is more suited to verbal content, whereas informative audio is better for numerical content, (iii) Constant audio triggering degrades learning performance and thus audio triggering should be handled with care. Based on these findings, we develop an ANN-based estimator to determine the proper moment for triggering audio (i.e. when memory performance is estimated to be declining) and carry out follow-up experiments for testing the integrated framework. Our contributions involve (i) determination of the most effective audio for each content type, (ii) estimation of memory deterioration based on learners' interaction logs, and (iii) the proposal of improvement of memory registration through auditory reinforcement. We believe that such findings constitute encouraging evidence the memory registration of e-learners can be enhanced with content-aware audio incorporation. en-copyright= kn-copyright= en-aut-name=SupitayakulParisa en-aut-sei=Supitayakul en-aut-mei=Parisa kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=1 ORCID= en-aut-name=YücelZeynep en-aut-sei=Yücel en-aut-mei=Zeynep kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=2 ORCID= en-aut-name=MondenAkito en-aut-sei=Monden en-aut-mei=Akito kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=3 ORCID= affil-num=1 en-affil=Graduate School of Natural Science and Technology, Okayama University kn-affil= affil-num=2 en-affil=Graduate School of Natural Science and Technology, Okayama University kn-affil= affil-num=3 en-affil=Graduate School of Natural Science and Technology, Okayama University kn-affil= en-keyword=Visualization kn-keyword=Visualization en-keyword=Electronic learning kn-keyword=Electronic learning en-keyword=Task analysis kn-keyword=Task analysis en-keyword=Estimation kn-keyword=Estimation en-keyword=Vocabulary kn-keyword=Vocabulary en-keyword=Memory management kn-keyword=Memory management en-keyword=Learning (artificial intelligence) kn-keyword=Learning (artificial intelligence) en-keyword=E-learning kn-keyword=E-learning en-keyword=neural networks kn-keyword=neural networks en-keyword=artificial intelligence kn-keyword=artificial intelligence en-keyword=cognitive theory of multimedia learning kn-keyword=cognitive theory of multimedia learning en-keyword=cognitive load kn-keyword=cognitive load en-keyword=distinctiveness account kn-keyword=distinctiveness account en-keyword=perceptual decoupling kn-keyword=perceptual decoupling en-keyword=adaptability kn-keyword=adaptability en-keyword=educational data mining kn-keyword=educational data mining END start-ver=1.4 cd-journal=joma no-vol=10 cd-vols= no-issue= article-no= start-page=134580 end-page=134591 dt-received= dt-revised= dt-accepted= dt-pub-year=2022 dt-pub=20221226 dt-online= en-article= kn-article= en-subject= kn-subject= en-title= kn-title=Neg/pos-Normalized Accuracy Measures for Software Defect Prediction en-subtitle= kn-subtitle= en-abstract= kn-abstract=In evaluating the performance of software defect prediction models, accuracy measures such as precision and recall are commonly used. However, most of these measures are affected by neg/pos ratio of the data set being predicted, where neg is the number of negative cases (defect-free modules) and pos is the number of positive cases (defective modules). Thus, it is not fair to compare such values across different data sets with different neg/pos ratios and it may even lead to misleading or contradicting conclusions. The objective of this study is to address the class imbalance issue in assessing performance of defect prediction models. The proposed method relies on computation of expected values of accuracy measures based solely on the value of the neg and pos values of the data set. Based on the expected values, we derive the neg/pos-normalized accuracy measures, which are defined as their divergence from the expected value divided by the standard deviation of all possible prediction outcomes. The proposed measures enable us to provide a ranking of predictions across different data sets, which can distinguish between successful predictions and unsuccessful predictions. Our results derived from a case study of defect prediction based on 19 defect data sets indicate that ranking of predictions is significantly different than the ranking of conventional accuracy measures such as precision and recall as well as composite measures F1-value, AUC of ROC, MCC, G-mean and Balance. In addition, we conclude that MCC attains a better defect prediction accuracy than F1-value, AUC of ROC, G-mean and Balance. en-copyright= kn-copyright= en-aut-name=GanMaohua en-aut-sei=Gan en-aut-mei=Maohua kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=1 ORCID= en-aut-name=YucelZeynep en-aut-sei=Yucel en-aut-mei=Zeynep kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=2 ORCID= en-aut-name=MondenAkito en-aut-sei=Monden en-aut-mei=Akito kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=3 ORCID= affil-num=1 en-affil=Graduate School of Natural Science and Technology, Okayama University kn-affil= affil-num=2 en-affil=Graduate School of Natural Science and Technology, Okayama University kn-affil= affil-num=3 en-affil=Graduate School of Natural Science and Technology, Okayama University kn-affil= en-keyword=Software defect kn-keyword=Software defect en-keyword=defect prediction model kn-keyword=defect prediction model en-keyword=accuracy measure kn-keyword=accuracy measure en-keyword=classification technology kn-keyword=classification technology en-keyword=empirical software engineering kn-keyword=empirical software engineering END start-ver=1.4 cd-journal=joma no-vol=10 cd-vols= no-issue= article-no= start-page=70053 end-page=70067 dt-received= dt-revised= dt-accepted= dt-pub-year=2022 dt-pub=2022 dt-online= en-article= kn-article= en-subject= kn-subject= en-title= kn-title=Improvement and Evaluation of Data Consistency Metric CIL for Software Engineering Data Sets en-subtitle= kn-subtitle= en-abstract= kn-abstract=Software data sets derived from actual software products and their development processes are widely used for project planning, management, quality assurance and process improvement, etc. Although it is demonstrated that certain data sets are not fit for these purposes, the data quality of data sets is often not assessed before using them. The principal reason for this is that there are not many metrics quantifying fitness of software development data. In that respect, this study makes an effort to fill in the void in literature by devising a new and efficient assessment method of data quality. To that end, we start as a reference from Case Inconsistency Level (CIL), which counts the number of inconsistent project pairs in a data set to evaluate its consistency. Based on a follow-up evaluation with a large sample set, we depict that CIL is not effective in evaluating the quality of certain data sets. By studying the problems associated with CIL and eliminating them, we propose an improved metric called Similar Case Inconsistency Level (SCIL). Our empirical evaluation with 54 data samples derived from six large project data sets shows that SCIL can distinguish between consistent and inconsistent data sets, and that prediction models for software development effort and productivity built from consistent data sets achieve indeed a relatively higher accuracy. en-copyright= kn-copyright= en-aut-name=GanMaohua en-aut-sei=Gan en-aut-mei=Maohua kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=1 ORCID= en-aut-name=YucelZeynep en-aut-sei=Yucel en-aut-mei=Zeynep kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=2 ORCID= en-aut-name=MondenAkito en-aut-sei=Monden en-aut-mei=Akito kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=3 ORCID= affil-num=1 en-affil=Graduate School of Natural Science and Technology, Okayama University kn-affil= affil-num=2 en-affil=Graduate School of Natural Science and Technology, Okayama University kn-affil= affil-num=3 en-affil=Graduate School of Natural Science and Technology, Okayama University kn-affil= en-keyword=Software kn-keyword=Software en-keyword=Measurement kn-keyword=Measurement en-keyword=Estimation kn-keyword=Estimation en-keyword=Data integrity kn-keyword=Data integrity en-keyword=Redundancy kn-keyword=Redundancy en-keyword=Data models kn-keyword=Data models en-keyword=Software engineering kn-keyword=Software engineering en-keyword=Data quality metric kn-keyword=Data quality metric en-keyword=data inconsistency kn-keyword=data inconsistency en-keyword=software project data analysis kn-keyword=software project data analysis en-keyword=software effort estimation kn-keyword=software effort estimation en-keyword=software productivity estimation kn-keyword=software productivity estimation END start-ver=1.4 cd-journal=joma no-vol=E103.D cd-vols= no-issue=10 article-no= start-page=2094 end-page=2103 dt-received= dt-revised= dt-accepted= dt-pub-year=2020 dt-pub=20201001 dt-online= en-article= kn-article= en-subject= kn-subject= en-title= kn-title=Empirical Evaluation of Mimic Software Project Data Sets for Software Effort Estimation en-subtitle= kn-subtitle= en-abstract= kn-abstract=To conduct empirical research on industry software development, it is necessary to obtain data of real software projects from industry. However, only few such industry data sets are publicly available; and unfortunately, most of them are very old. In addition, most of today's software companies cannot make their data open, because software development involves many stakeholders, and thus, its data confidentiality must be strongly preserved. To that end, this study proposes a method for artificially generating a “mimic” software project data set, whose characteristics (such as average, standard deviation and correlation coefficients) are very similar to a given confidential data set. Instead of using the original (confidential) data set, researchers are expected to use the mimic data set to produce similar results as the original data set. The proposed method uses the Box-Muller transform for generating normally distributed random numbers; and exponential transformation and number reordering for data mimicry. To evaluate the efficacy of the proposed method, effort estimation is considered as potential application domain for employing mimic data. Estimation models are built from 8 reference data sets and their concerning mimic data. Our experiments confirmed that models built from mimic data sets show similar effort estimation performance as the models built from original data sets, which indicate the capability of the proposed method in generating representative samples. en-copyright= kn-copyright= en-aut-name=GanMaohua en-aut-sei=Gan en-aut-mei=Maohua kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=1 ORCID= en-aut-name=YücelZeynep en-aut-sei=Yücel en-aut-mei=Zeynep kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=2 ORCID= en-aut-name=MondenAkito en-aut-sei=Monden en-aut-mei=Akito kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=3 ORCID= en-aut-name=SasakiKentaro en-aut-sei=Sasaki en-aut-mei=Kentaro kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=4 ORCID= affil-num=1 en-affil=Okayama University kn-affil= affil-num=2 en-affil=Okayama University kn-affil= affil-num=3 en-affil=Okayama University kn-affil= affil-num=4 en-affil=Okayama University kn-affil= en-keyword=empirical software engineering kn-keyword=empirical software engineering en-keyword=data confidentiality kn-keyword=data confidentiality en-keyword=data mining kn-keyword=data mining END start-ver=1.4 cd-journal=joma no-vol=E103.D cd-vols= no-issue=8 article-no= start-page=1865 end-page=1874 dt-received= dt-revised= dt-accepted= dt-pub-year=2020 dt-pub=20200801 dt-online= en-article= kn-article= en-subject= kn-subject= en-title= kn-title=An Algorithm for Automatic Collation of Vocabulary Decks Based on Word Frequency en-subtitle= kn-subtitle= en-abstract= kn-abstract=This study focuses on computer based foreign language vocabulary learning systems. Our objective is to automatically build vocabulary decks with desired levels of relative difficulty relations. To realize this goal, we exploit the fact that word frequency is a good indicator of vocabulary difficulty. Subsequently, for composing the decks, we pose two requirements as uniformity and diversity. Namely, the difficulty level of the cards in the same deck needs to be uniform enough so that they can be grouped together and difficulty levels of the cards in different decks need to be diverse enough so that they can be grouped in different decks. To assess uniformity and diversity, we use rank-biserial correlation and propose an iterative algorithm, which helps in attaining desired levels of uniformity and diversity based on word frequency in daily use of language. In experiments, we employed a spaced repetition flashcard software and presented users various decks built with the proposed algorithm, which contain cards from different content types. From users' activity logs, we derived several behavioral variables and examined the polyserial correlation between these variables and difficulty levels across different word classes. This analysis confirmed that the decks compiled with the proposed algorithm induce an effect on behavioral variables in line with the expectations. In addition, a series of experiments with decks involving varying content types confirmed that this relation is independent of word class. en-copyright= kn-copyright= en-aut-name=YücelZeynep en-aut-sei=Yücel en-aut-mei=Zeynep kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=1 ORCID= en-aut-name=SupitayakulParisa en-aut-sei=Supitayakul en-aut-mei=Parisa kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=2 ORCID= en-aut-name=MondenAkito en-aut-sei=Monden en-aut-mei=Akito kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=3 ORCID= en-aut-name=LeelaprutePattara en-aut-sei=Leelaprute en-aut-mei=Pattara kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=4 ORCID= affil-num=1 en-affil=Okayama University kn-affil= affil-num=2 en-affil=Okayama University kn-affil= affil-num=3 en-affil=Okayama University kn-affil= affil-num=4 en-affil=Department of Computer Engineering, Faculty of Engineering, Kasetsart University kn-affil= en-keyword=e-learning kn-keyword=e-learning en-keyword=vocabulary learning kn-keyword=vocabulary learning en-keyword=log file analysis kn-keyword=log file analysis END start-ver=1.4 cd-journal=joma no-vol=36 cd-vols= no-issue=16 article-no= start-page=1527 end-page=1539 dt-received= dt-revised= dt-accepted= dt-pub-year=2020 dt-pub=20200526 dt-online= en-article= kn-article= en-subject= kn-subject= en-title= kn-title=Estimating Level of Engagement from Ocular Landmarks en-subtitle= kn-subtitle= en-abstract= kn-abstract=E-learning offers many advantages like being economical, flexible and customizable, but also has challenging aspects such as lack of – social-interaction, which results in contemplation and sense of remoteness. To overcome these and sustain learners’ motivation, various stimuli can be incorporated. Nevertheless, such adjustments initially require an assessment of engagement level. In this respect, we propose estimating engagement level from facial landmarks exploiting the facts that (i) perceptual decoupling is promoted by blinking during mentally demanding tasks; (ii) eye strain increases blinking rate, which also scales with task disengagement; (iii) eye aspect ratio is in close connection with attentional state and (iv) users’ head position is correlated with their level of involvement. Building empirical models of these actions, we devise a probabilistic estimation framework. Our results indicate that high and low levels of engagement are identified with considerable accuracy, whereas medium levels are inherently more challenging, which is also confirmed by inter-rater agreement of expert coders. en-copyright= kn-copyright= en-aut-name=YucelZeynep en-aut-sei=Yucel en-aut-mei=Zeynep kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=1 ORCID= en-aut-name=KoyamaSerina en-aut-sei=Koyama en-aut-mei=Serina kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=2 ORCID= en-aut-name=MondenAkito en-aut-sei=Monden en-aut-mei=Akito kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=3 ORCID= en-aut-name=SasakuraMariko en-aut-sei=Sasakura en-aut-mei=Mariko kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=4 ORCID= affil-num=1 en-affil=Department of Computer Science, Division of Industrial Innovation Sciences, Okayama University kn-affil= affil-num=2 en-affil=Department of Computer Science, Division of Industrial Innovation Sciences, Okayama University kn-affil= affil-num=3 en-affil=Department of Computer Science, Division of Industrial Innovation Sciences, Okayama University kn-affil= affil-num=4 en-affil=Department of Computer Science, Division of Industrial Innovation Sciences, Okayama University kn-affil= END start-ver=1.4 cd-journal=joma no-vol=19 cd-vols= no-issue=2 article-no= start-page=58 end-page=64 dt-received= dt-revised= dt-accepted= dt-pub-year=2017 dt-pub=201703 dt-online= en-article= kn-article= en-subject= kn-subject= en-title= kn-title=Probing Software Engineering Beliefs about System Testing Defects: Analyzing Data for Future Directions en-subtitle= kn-subtitle= en-abstract= kn-abstract= Research findings are often expressed as short startling sentences or software engineering (SE) beliefs such as “about 80 percent of the defects come from 20 percent of the modules” and “peer reviews catch 60 percent of the defects” [2]. Such SE beliefs are particularly important in industry, as they are attention-getting, easily understandable, and thus practically useful. In this paper we examine the power of such SE beliefs to justify process improvement through empirical validation of selected beliefs related to the increase or decrease of defects in system testing. We explore four basic SE beliefs in data from two midsize embedded software development organizations in Japan, and based on this information, identify possible process improvement actions for each organization. Based on our study, even small and medium-sized enterprises (SMEs) can use this approach to find possible directions to improve their process, which will result in better products. en-copyright= kn-copyright= en-aut-name=MondenAkito en-aut-sei=Monden en-aut-mei=Akito kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=1 ORCID= en-aut-name=TsunodaMasateru en-aut-sei=Tsunoda en-aut-mei=Masateru kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=2 ORCID= en-aut-name=BarkerMike en-aut-sei=Barker en-aut-mei=Mike kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=3 ORCID= en-aut-name=MatsumotoKenichi en-aut-sei=Matsumoto en-aut-mei=Kenichi kn-aut-name= kn-aut-sei= kn-aut-mei= aut-affil-num=4 ORCID= affil-num=1 en-affil=Graduate School of Natural Science and Technology, Okayama University kn-affil= affil-num=2 en-affil=Department of Informatics, Faculty of Science and Engineering, Kindai University kn-affil= affil-num=3 en-affil=Graduate School of Information Science, Nara Institute of Science and Technology kn-affil= affil-num=4 en-affil=Graduate School of Information Science, Nara Institute of Science and Technology kn-affil= en-keyword=D.2.19 Software Quality/SQA kn-keyword=D.2.19 Software Quality/SQA en-keyword=D.2.8.c Process metrics kn-keyword=D.2.8.c Process metrics END