- Original Research
- Open Access
Thought Chart: tracking the thought with manifold learning during emotion regulation
- Mengqi Xing^{1},
- Johnson GadElkarim^{2},
- Olusola Ajilore^{3},
- Ouri Wolfson^{2},
- Angus Forbes^{2},
- K. Luan Phan^{3},
- Heide Klumpp^{3} and
- Alex Leow^{4}Email author
- Received: 10 February 2017
- Accepted: 12 July 2018
- Published: 19 July 2018
Abstract
The Nash embedding theorem demonstrates that any compact manifold can be isometrically embedded in a Euclidean space. Assuming the complex brain states form a high-dimensional manifold in a topological space, we propose a manifold learning framework, termed Thought Chart, to reconstruct and visualize the manifold in a low-dimensional space. Furthermore, it serves as a data-driven approach to discover the underlying dynamics when the brain is engaged in a series of emotion and cognitive regulation tasks. EEG-based temporal dynamic functional connectomes are created based on 20 psychiatrically healthy participants’ EEG recordings during resting state and an emotion regulation task. Graph dissimilarity space embedding was applied to all the dynamic EEG connectomes. In order to visualize the learned manifold in a lower dimensional space, local neighborhood information is reconstructed via k-nearest neighbor-based nonlinear dimensionality reduction (NDR) and epsilon distance-based NDR. We showed that two neighborhood constructing approaches of NDR embed the manifold in a two-dimensional space, which we named Thought Chart. In Thought Chart, different task conditions represent distinct trajectories. Properties such as the distribution or average length in the 2-D space may serve as useful parameters to explore the underlying cognitive load and emotion processing during the complex task. In sum, this framework is a novel data-driven approach to the learning and visualization of underlying neurophysiological dynamics of complex functional brain data.
Keywords
- Thought Chart
- Nonlinear dimensionality reduction
- Nearest neighbor
- Manifold learning
- EEG connectome
- Emotion regulation
1 Background
The Nash embedding theorems [1, 2] showed that any Riemannian n-manifold with a \(C^{1}\) positive metric has an isometric embedding in a Euclidean space of dimension 2n+1, even in any small portion of this space. Since the Gaussian curvature of a surface is invariant under local isometry based on the Theorema Egregium [3], the manifold properties in a low-dimensional space can provide an insight into the topological structure of the brain. Recent advent in structural and functional neuroimaging has further suggested that brain imaging data may construct a smooth and differentiable manifold, in which local neighborhood properties can assist in recognizing the underlying pattern in trajectories of brain development [4], discriminating different types of brain tumors [5], and improving the imaging registration accuracy [6]. However, the manifold associated with temporally varying brain dynamics imaging data has not yet been explored. Here, we propose a framework to examine the intrinsic geometry of the mind’s topological space of functional brain imaging via dissimilarity-based manifold learning.
We assume brain states compose a high-dimensional space [4, 7], which can be reconstructed and visualized in a low-dimensional space via dimensionality reduction. Common dimensionality reduction approaches including principal component analysis (PCA) and linear discriminant analysis (LDA) are often used in functional brain networks to visualize select regions of interest [8, 9]. However, a linear approach cannot recognize nonlinear structures in a high-dimensional space [9], making it unsuitable for preserving the global intrinsic geometry for complex dynamic brain data. Nonlinear dimensionality reduction approach isomap yields global coordinates which provide a simple way to analyze and manipulate high-dimensional observations in terms of their intrinsic nonlinear degrees of freedom and produce a globally optimal low-dimensional Euclidean representation [10].
The functional network data are informed by electroencephalogram (EEG) for its high temporal resolution, coupled with the emotion regulation task. The synchronization of response systems is highly dynamic when human brains are engaging in emotion and cognitive tasks [11]. Thus, with proper sectioning, the synchronicity between regions of a brain network can be described, in the form of a connectome, using a phase-based connectivity analyses approach, weighted phase lag index (WPLI) [12]. Guided by our own published research results, theta wave may be the most sensitive among all EEG frequencies in investigating EEG networks in emotion regulation task (ERT) [13, 14]. Therefore, we selected theta wave network connectivity as the input for our manifold learning.
Here, we propose a manifold learning framework, Thought Chart, to explore the underlying intrinsic geometry of dynamic functional networks. The hypothesis of this manifold learning is that the reconstructed manifold will reflect different properties of the brain’s state during tasks. Additionally, by sampling the space, we can extract specific aspects of the trajectory in this manifold that reflect task performance, such as coordinates and their distribution and levels of scattering [15]. To recover the local neighborhood information needed for the nonlinear dimensionality reduction (NDR) step, here we tested two strategies in searching for local neighbors: K-nearest neighbor (KNN), which sets each point to search for its k-nearest points; and epsilon radius, which sets each point to search for all points within a fixed radius \(\varepsilon\). Properties of Thought Chart constructed by these two approaches will be evaluated to see how neighborhood identification influences the intrinsic geometry of human brain.
2 Methods
2.1 Data acquisition and emotion regulation tasks (ERT)
2.2 Weighted phase lag index-based EEG connectome
2.3 Learning the manifold with graph dissimilarity space embedding
To collect sufficient amount of data points to learn the intrinsic geometry of a high-dimensional manifold, we utilized the EEG connectomes from all subjects at all time points as sampling possible states of the manifold that is shared among all subjects. Then, graph dissimilarity space embedding is used to represent each connectome as a point in a very high-dimensional space (over \(10^{4}\) ), which is described below [18, 19]. Assuming labeled sample graph set \(G={G_{1},..., G_{n} }\) has n ”prototype” graph observations \(G_{i} \in \mathbb {G}\) (the set of all possible graphs under consideration) and d is the distance metric that can be computed between two graphs \(d: \mathbb {G} \times \mathbb {G} \rightarrow [0,\infty )\), then any graph \(X \in \mathbb {G}\) can be represented using \(\varphi _{n}^{G}:\mathbb {G} \rightarrow \mathbb {R}n\), defined as the n-dimensional vector \(\varphi _{n}^{G}(X) = [d(X, G_{1}), ... d(X, G_{n})]\). In this way, any graph set can be represented by a set of real numbers. In our case, all the connectomes were initially used as prototypes for an unsupervised learning. Thus, the number of dimensions is in the same order as the number of observations in the dataset.
Reconstructing the local neighborhood reconstruction. Here, we emphasize that this step is crucial in order to properly learn the manifold’s intrinsic geometry, as d (which is used to define coordinates in the embedding space, and thus not intrinsic to the manifold) will not properly inform geodesics (the shortest paths on the manifold, which is an intrinsic property) except in local neighborhoods. While such a construction calls for a “good” choice of the distance function d, we posit that given a sufficiently large amount of data points the learned manifold will converge to the true manifold with any reasonably chosen d. Given two connectome matrices X and Y, a natural choice, which we adopted here, is the Euclidean distance: \(d(X,Y)=\sqrt{\sum _{ij}(X_{ij}-Y_{ij})^{2} }\) and \(\left\| \varphi _{n}^{G}(X)-\varphi _{n}^{G}(Y)\right\| =\sqrt{\sum _{k}(d(Y,G_{k})-d(X,G_{k}))^{2}}\) .
2.4 Nonlinear dimensionality reduction
Once the local neighborhood is learned, a matrix representing this high-dimensional manifold was reconstructed into a lower dimensional Euclidean space via NDR. Once this is achieved, Thought Chart of any given individual can be constructed by tracing the trajectory of the time-dependent connectome of that subject for any given task. In this case, we selected isomap, a nonlinear prototypical isometric embedding procedure which entails the computation of geodesics based on neighborhood information followed by the (quasi-)isometric embedding of the geodesics.
To provide a good approximation to geodesic distance, the first step of isomap is to determine the neighbors of each point on the low-dimensional manifold based on the distance matrix acquired from the previous step [20]. Here, we compared two common approaches to determine whether two points are neighbors: the k-isomap method and the \(\varepsilon -isomap\) method. K-isomap utilized the k-nearest neighbor algorithm to determine neighbors, while \(\varepsilon\)-isomap includes all the points within some fixed radius \(\varepsilon\). The relationships of the neighborhood are represented in a weighted graph D in which \(D(X, Y) = d(X, Y)\) if X and Y are neighbors, otherwise\(d(X, Y) = \infty\) [20]. The second step relies on applying the classic multidimensional scaling (MDS) to the centered squared geodesic distance matrix, whose eigendecomposition provides the basis for lower dimensional embedding.
2.5 Exploration of ERT Thought Chart
3 Results
3.1 Thought Chart construction
3.2 K-isomap
To further understand theta EEG connectome dynamics, we additionally studied the four distinct subregions of the manifold (i.e., segments of the “snake”): the head (primarily resting), the mid body (primarily Neutral), the posterior body (a mixture of Neutral, Maintain, and Reappraise) and the tail (primarily Maintain and Reappraise; Fig. 4). Sampling these segments reveals marked connectome differences. Analysis of the top ten edge strengths in the head region (Fig. 4a) demonstrated increased theta coupling in fronto-parieto-occipital leads while the body (Neutral-predominant, Fig. 4b; Maintain/Reappraise dominant, Fig. 4c) is characterized by predominant theta coupling between occipital leads. Last, the tail (Maintain/Reappraise only, Fig. 4d) revealed increased theta coupling between frontal and parietal leads.
3.3 ε-isomap
Thought Chart of the \(\varepsilon\)-isomap presents a similar distribution along the principle dimension. Taking samples for the head, mid body, posterior body, and tail, the overall connectivity of the average connectome is increasing monotonically, which is different from k-isomap. The top ten edge strengths in the head region (Fig. 5a) demonstrated strong parieto-central and occipital theta coupling, while the body (Fig. 5b, c) presented increasing theta coupling within occipital channels. Lastly, the increased theta coupling between frontal and occipital leads can be observed at the tail (Fig. 5d).
3.4 Average trajectory length and spreadness
In k-isomap Thought Chart, average trajectory length in the Euclidean space was significantly longer in more complex tasks (Fig. 6a), (one-way ANOVA: \(df=2, F=19.60, \,p<0.001\), paired t test: Neutral vs. Maintain: \(p=0.008\), Maintain vs. Reappraise: \(p=0.004\), Neutral vs. Reappraise \(p<0.001\)). In \(\varepsilon\)-isomap Thought Chart, a similar trend is observed, where subjects tend to “travel” longer distance in Thought Chart during tasks with higher cognitive demand (Fig. 6b). (One-way ANOVA: \(df=2,\, F=21.75, \, p<0.001\), paired t test: Neutral vs. Maintain: \(p=0.002\), Maintain vs. Reappraise: \(p=0.003\), Neutral vs. Reappraise \(p<0.001\).) Furthermore, task differences not only were reflected in the average distance traveled, but also observed in the level of scattering measured using spreadness (Fig. 7), in both k-isomap (one-way ANOVA: \(df=2, F=15.61 \,p<0.001,\) paired t test: Neutral vs. Maintain: \(p=0.015\), Maintain vs. Reappraise: \(p=0.047\), Neutral vs. Reappraise \(p<0.001\) ) and \(\varepsilon\)-isomap (one-way ANOVA: \(df=2, F=19.20,p<0.001\), paired t test: Neutral vs. Maintain: \(p=0.008\), Maintain vs. Reappraise: \(p=0.017\), Neutral vs. Reappraise \(p<0.001\).)
4 Discussion
In this study, we proposed a novel unsupervised manifold learning framework to construct a state space, in the form of a manifold embedded in 2-D that quasi-isometrically visualizes EEG connectome dynamics. Moreover, in this space one can visualize time-dependent brain activities as a trajectory or Thought Chart. Using the temporal dynamic EEG connectome, two neighborhood construction algorithms are applied to visualize the state space in 2-D. Both k-isomap and \(\varepsilon\)-isomap are able to compose a highly dynamic and complex geometry, where distinct subregions are distributed along the principle dimension (x-axis in the 2-D space). The baseline resting state is concentrated on one end, followed by mostly Neutral points that can be observed later along the x-axis, while Maintain and Reappraise are mixed, from the posterior section on to the end. Note that this transition corresponds to different mental states, suggesting that the manifold has a principal dimension that is primarily linear, and a vertical motion around this principal dimension whose amplitude increases with cognitive demands. In this context, this manifold may resemble dynamical systems on a torus [22] (the surface of a doughnut), in that trajectories are generated by the product of distribution along the principal dimension and a minor up–down motion in the second dimension.
By sectioning the region where different tasks concentrated, matrices can be defined to represent these regions. As similar Thought Chart trajectories are, regional matrices of k-isomap and \(\varepsilon\)-isomap however can be different. There are few possible explanations. First, even if each region appears to have a similar composition of tasks, due to the different neighborhood constructing strategy, the points being included in the same regions are not necessarily the same. Second, the sectioning of the head, body, and tail is arbitrary; hence, it can only provide an approximate matrix representation without rigorously controlling the number of points of this region. However, the strongest theta couplings in each region are similar between two methods. Based on the strongest coupling, the manifold comprises subspaces representing resting, visual processing (a common feature of Neutral, Maintain, and Reappraise), and cognitive control (a distinct feature of Reappraise). Edge strength analyses of the manifold-sampled EEG connectomes demonstrated increased patterns of theta coupling that are highly consistent with previous reports of frequency-band coupling associated with the resting state [23], visual processing [24], and cognitive control [25].
As it is shown in Figs. 2 and 3, Thought Chart is able to visualize an individual’s “mind travels” in a two-dimensional space in this 7-s session. It thus allows us to quantify the temporal dynamics of EEG brain networks by looking into properties such as the distribution and trajectory length. For healthy participants, the mind is more or less “static” when the task is simple and becomes more dynamic as the task requires higher cognitive demand. These properties can be extracted as biomarkers for mind state predictions in future applications.
Limitations of our approach merit further discussion. First, the parameter setting for \(\varepsilon\)-isomap can be very sensitive since \(\varepsilon\) is defined as a distance which can be tuned to go either fine (local) or coarse (global) (Fig. 8b). In this dataset, the reconstructed trajectory is stable as the \(\varepsilon\) changed within a certain range. However, in future applications, it may create further complexity in determining the optimal \(\varepsilon\), especially if the data are sensitive toward \(\varepsilon\). Secondly, it may be inappropriate to compare these thought chart metrics in the resting state with those from Neutral, Maintain, and Reappraise. This is due to the nature of a task-free design (resting state) versus a task-related design during ERT. Unlike in the resting state when research participants were simply instructed to do nothing (and thus leading to mind wandering), in ERT there is a clear “begin” and “end” cue for all Neutral, Maintain, and Reappraise sessions and participants were instructed to accomplish a certain task between the cues. Thus, we are able to precisely calculate trajectories by averaging across trials and compare them across three task conditions. However, the resting-state data were collected from a continuous session, during which participants were not instructed to think in any particular way. Therefore, the resting-state data were only included as “baseline reference” in dissimilarity embedding and in isomap visualization but not during subsequent quantitative analyses. Moreover, as a quasi-isometric technique isomap aims to preserve the pairwise geodesics on the manifold, i.e., approximating global isometry when the embedding is constrained to a given dimension. By contrast, other classes of local NDR methods such as LLE unfold the manifold by preserving local linear reconstruction relationship (i.e., local parameterization) of each point within its neighborhood. Furthermore, as the Theorema Egregium only guaranteed the invariance of Gauss curvature for complete isometric embeddings of two manifolds, it is unclear whether the manifold constructed using one NDR technique is necessarily more “correct.” Nevertheless, both LLE and isomap recover a principal dimension and a up–down motion around it, while simple linear techniques such as PCA did not. We thus posit that the highly structured complex geometry recovered using our framework may indeed inform the hidden properties of brain dynamics and the underlying neurophysiological mechanisms that generate them.
Declarations
Authors' contributions
MX processed the EEG data and completed the network estimation. EEG data were collected under the administration of HK. AL, OA, and OW constructed the dissimilarity embedding and non-dimensionality reduction framework. MX and JG wrote the complete Thought Chart scripts in Matlab. MX, JG, OA, AL, AF, HK, and KLP drafted and revised the manuscript together. All authors read and approved the final manuscript.
Acknowlegdements
This work was supported by NIMH K23MH093679 (HK) and in part by NIMH R01MH101497 (KLP) and the Center for Clinical and Translational Research (CCTS) UL1RR029879, and University of Illinois at Chicago Campus Research Board Award.
Competing interests
The authors declare that they have no competing interests.
Ethics approval and consent to participate
All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.
Consent for publication
Informed consent was obtained from all individual participants included in the study.
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.
Authors’ Affiliations
References
- Nash J (1954) C1 isometric imbeddings. Ann Math 60:383–396MathSciNetView ArticleMATHGoogle Scholar
- Nash J (1956) The imbedding problem for Riemannian manifolds. Ann Math 63:20–63MathSciNetView ArticleMATHGoogle Scholar
- Gauss CF (1902) General investigations of curved surfaces of 1827 and 1825. Princeton University Library, PrincetonMATHGoogle Scholar
- Aljabar P, Wolz R, Srinivasan L, Counsell SJ, Rutherford MA, Edwards AD, Hajnal JV, Rueckert D (2011) A combined manifold learning analysis of shape and appearance to characterize neonatal brain development. IEEE Trans Med Imaging 30:2072–2086View ArticleGoogle Scholar
- Cruz-Barbosa R, Vellido A (2011) Semi-supervised analysis of human brain tumors from partially labeled MRS information, using manifold learning models. Int J Neural Syst 21:17–29View ArticleGoogle Scholar
- Ye DH, Hamm J, Kwon D, Davatzikos C, Pohl, KM (2012) Regional manifold learning for deformable registration of brain MR images. In: Medical image computing and computer-assisted intervention $$-$$ - MICCAI 2012: 15th international conference, Nice, France, October 1–5, 2012, proceedings, part III, pp 131–138. Springer, BerlinGoogle Scholar
- Poldrack RA, Halchenko YO, Hanson SJ (2009) Decoding the large-scale structure of brain function by classifying mental states across individuals. Psychol Sci 20:1364–1372View ArticleGoogle Scholar
- Zhou Z, Chen Y, Ding M, Wright P, Lu Z, Liu Y (2009) Analyzing brain networks with PCA and conditional Granger causality. Hum Brain Mapp 30:2197–2206View ArticleGoogle Scholar
- Lotte F, Congedo M, Lécuyer A, Lamarche F, Arnaldi B (2007) A review of classification algorithms for EEG-based brain-computer interfaces. J Neural Eng 4:R1View ArticleGoogle Scholar
- Tenenbaum JB, Silva VD, Langford JC (2000) A global geometric framework for nonlinear dimensionality reduction. Science 290:2319–2323View ArticleGoogle Scholar
- Scherer KR (2009) The dynamic architecture of emotion: Evidence for the component process model. Cognit Emot 23:1307–1351View ArticleGoogle Scholar
- Cohen MX (2013) Analyzing neural time series data: theory and practice. The MIT Press, LondonGoogle Scholar
- Xing M, Tadayonnejad R, MacNamara A, Ajilore O, DiGangi J, Phan KL, Leow A, Klumpp H (2017) Resting-state theta band connectivity and graph analysis in generalized social anxiety disorder. NeuroImage: Clin 13:24–32View ArticleGoogle Scholar
- Xing M, Tadayonnejad R, MacNamara A, Ajilore O, Phan KL, Klumpp H, Leow A (2016) EEG based functional connectivity reflects cognitive load during emotion regulation. In: 2016 IEEE 13th international symposium on biomedical imaging (ISBI), pp. 771–774Google Scholar
- Xing M, Ajilore O, Wolfson O, Abbott C, MacNamara A, Tadayonnejad R, Forbes A, Phan KL, Klumpp H, Leow A (2016) Thought chart: tracking dynamic EEG brain connectivity with unsupervised manifold learning. In: International conference on brain and health informatics, pp 149–157Google Scholar
- Parvaz MA, MacNamara A, Goldstein RZ, Hajcak G (2012) Event-related induced frontal alpha as a marker of lateral prefrontal cortex activation during cognitive reappraisal. Cognit Affect Behavi Neurosci 12:730–740View ArticleGoogle Scholar
- Gross JJ (1998) The emerging field of emotion regulation: an integrative review. Rev General Psychol 2:271–299View ArticleGoogle Scholar
- Bunke H, Riesen K (2008) Graph Classification Based on Dissimilarity Space Embedding. In: da Vitoria Lobo, N, Kasparis T, Roli F, Kwok JT, Georgiopoulos M, Anagnostopoulos GC, Loog M (eds) Structural, syntactic, and statistical pattern recognition: joint IAPR international workshop, SSPR and SPR 2008, Orlando, USA, December 4–6, 2008. Proceedings, pp 996–1007. Springer, BerlinGoogle Scholar
- Duin RPW, Loog M, Pekalska E, Tax DMJ (2010) Feature-based dissimilarity space classification. In: Unay D, Çataltepe Z, Aksoy S (eds) Recognizing patterns in signals, speech, images and videos: ICPR 2010 contests, Istanbul, Turkey, August 23–26, 2010, contest reports. Springer, Berlin, pp 46–55Google Scholar
- Yiming W, Kap Luk C, An extended Isomap algorithm for learning multi-class manifold. In: Proceedings of 2004 international conference on machine learning and cybernetics, vol 3426 (IEEE Cat. No.04EX826), pp 3429–3433Google Scholar
- Roweis ST, Saul LK (2000) Nonlinear dimensionality reduction by locally linear embedding. Science 290:2323–2326View ArticleGoogle Scholar
- Tozzi A, Peters JF (2016) Towards a fourth spatial dimension of brain activity. Cognit Neurodyn 10:189–199View ArticleGoogle Scholar
- Albrecht MA, Roberts G, Price G, Lee J, Iyyalol R, Martin-Iverson MT (2016) The effects of dexamphetamine on the resting-state electroencephalogram and functional connectivity. Hum Brain Mapp 37:570–588View ArticleGoogle Scholar
- Keil A, Costa V, Smith JC, Sabatinelli D, McGinnis EM, Bradley MM, Lang PJ (2012) Tagging cortical networks in emotion: a topographical analysis. Human Brain Mapp 33:2920–2931View ArticleGoogle Scholar
- Griesmayr B, Berger B, Stelzig-Schoeler R, Aichhorn W, Bergmann J, Sauseng P (2014) EEG theta phase coupling during executive control of visual working memory investigated in individuals with schizophrenia and in healthy controls. Cognit Affect Behav Neurosci 14:1340–1355View ArticleGoogle Scholar