Can ChatGPT Truly Think Sustainably? Evaluating Its SDG Intelligence

Redacción HC
25/04/2024

As artificial intelligence tools like ChatGPT become increasingly integrated into education, policymaking, and global discourse, a key question arises: is ChatGPT merely literate about the United Nations Sustainable Development Goals (SDGs), or does it possess the deeper intelligence needed to think sustainably? A new study published in PLOS ONE explores this question through a rigorous evaluation of ChatGPT's performance on two standardized SDG tests.

Understanding the Stakes: Literacy vs. Intelligence

The SDGs, a universal framework adopted by the UN to achieve a better and more sustainable future, require more than rote memorization. They demand systems thinking, strategic planning, and normative reasoning. As the influence of AI grows, so does the need to determine whether tools like ChatGPT are capable of supporting these cognitive demands.

The study, led by researchers from Amrita Vishwa Vidyapeetham and the Fortune Institute of International Business in India, investigates whether ChatGPT—specifically GPT-4—demonstrates genuine competence in understanding and applying the SDGs. In their words: "Does ChatGPT exhibit not only literacy about the SDGs, but also intelligence in reasoning with them?"

How the Study Was Conducted

Researchers tested ChatGPT using two globally recognized SDG evaluation tools:

1. SDG Fitness Test (UN SDG Learn)

This assessment gauges eight transversal competencies essential for sustainable thinking, including systems thinking, strategic analysis, collaboration, critical thinking, and anticipation. ChatGPT responded to 24 real-world scenario questions.

2. SULITEST

A broader literacy tool measuring factual understanding of sustainability through 30 multiple-choice questions.

In both cases, researchers ensured each question was fed to GPT-4 in isolation, with the chat history reset each time to avoid context accumulation. They then evaluated ChatGPT's overall scores, mapped responses against the 17 SDGs, and analyzed the model's consistency in displaying the core competencies.

Key Findings: High Knowledge, Uneven Reasoning

Strong Performance on SDG Literacy

• SULITEST Score: ChatGPT achieved a striking 97.5%, showcasing robust factual knowledge.
• SDG Fitness Test Score: It earned 85%, performing at an advanced level in normative and strategic competencies, and at an intermediate level in the remaining six.

These results suggest ChatGPT can serve as an effective SDG explainer or assistant in educational settings. However, a deeper dive reveals several limitations in its capacity to reason sustainably.

Gaps in Contextual Understanding

Despite high scores, the study uncovered inconsistencies that indicate limitations in ChatGPT's reasoning:

• Responses often misclassified which SDG applied to a given scenario.
• Certain goals—such as SDG 6 (clean water), SDG 12 (responsible consumption), SDG 14 (life below water), and SDG 15 (life on land)—were underrepresented or misunderstood.
• ChatGPT's answers were not always internally consistent or context-aware, suggesting a lack of stable conceptual understanding.

In short, while ChatGPT can recite what SDGs are, it doesn't always think with them reliably.

Competency Mapping

Among the eight core sustainability competencies, ChatGPT excelled in collaboration, systems thinking, and integrated problem-solving, but lagged in self-awareness and strategic foresight. This discrepancy reinforces the notion that the model's outputs are based more on pattern recognition than on deep understanding.

Why This Matters: Practical Implications

Education

ChatGPT could serve as a factual companion in SDG education—useful for quizzes, overviews, and introductory content. However, educators must supervise its use when teaching strategic thinking or ethical reasoning.

"ChatGPT is like an advanced dictionary, not a sustainable travel guide," the authors note. It can define the terrain, but not necessarily help navigate it thoughtfully.

Policy and Public Communication

The model's intermediate-to-advanced proficiency suggests it may assist in creating discussion materials, framing questions, or generating summaries. Still, human oversight remains essential, especially when nuance or moral reasoning is required.

Toward More Sustainable AI

The researchers advocate for AI that doesn't just mimic human knowledge but incorporates reflective, contextual, and evaluative capacities. Recommendations include:

• Adapting SDG tests to be more AI-friendly and better suited to measuring synthetic understanding.
• Improving consistency mechanisms in large language models (LLMs) to avoid contradictions and enhance goal alignment.
• Adding critical-thinking layers to model architecture or training pipelines, especially when dealing with complex global challenges.

A Call to Action: Rethinking AI for Global Goals

This study is among the first to quantitatively assess a large language model using standardized sustainability frameworks. It provides compelling evidence that while ChatGPT is highly literate in sustainability language, its intelligence is still developing.

As we integrate AI into classrooms, policy spaces, and civic discourse, we must ask: are we enabling tools that reinforce surface-level knowledge, or fostering systems that can reason for a better world?

Educators, policymakers, and developers should collaborate to build the next generation of AI tools that are not only informed but also aligned with the ethical, strategic, and systemic demands of sustainable development.

Topics of interest

Technology