Redundant natural language and irrelevant information also can hamper the performance of LLMs.
Using symbols to represent spatial relationships in word sequences can provide a clearer and more understandable framework for large language models (LLMs).
This method potentially simplifies data processing for the models.
Researchers are exploring the use of symbols in prompting LLMs, highlighting its importance as it extends beyond traditional language comprehension. This under-explored area offers valuable insights into enhancing LLM capabilities and understanding.
Using a set of symbols to represent spatial relationships between objects in complex environments.
To examine the impact of symbolic representations in prompting, researchers focus on complex spatial understanding and planning tasks.
These tasks require large language models (LLMs) to interpret virtual spatial environments described in natural language, while also planning and achieving specific objectives within those settings.
This approach helps assess how effectively LLMs can navigate and manipulate spatial information.
The model is posed with the following question:
Answer this question: I have a chair, two potatoes, a cauliflower, a lettuce head, two tables, a cabbage, two onions, and three fridges. How many vegetables do I have?
Symbolic reasoning involves reasoning about objects, their relation to each-other and concepts.
And The Model Response:
The question lists several items, but only some of them are vegetables. Here’s the breakdown:
Potatoes: 2 (vegetable)Cauliflower: 1 (vegetable)Lettuce head: 1 (vegetable)Cabbage: 1 (vegetable)Onions: 2 (vegetable)
The other items (chair, tables, and fridges) are not vegetables.
Total number of vegetables: 2 + 1 + 1 + 1 + 2 = 7.
So, you have 7 vegetables.
Spatial reasoning in natural language processing focuses on understanding and processing spatial information within text. Some researchers have explored methods for training models on navigation tasks that require agents to reach specific locations based on textual descriptions.
Others have developed tasks where models generate instructions from “before” and “after” image pairs or created benchmarks for answering spatial questions about environments. Additionally, it has been observed that large language models struggle with text-based games that involve multi-step reasoning.
