--- Forwarded Message from Ted Bell <[log in to unmask]> ---

>Date: 5 Sep 2001 15:40:51 -0700
>To: [log in to unmask]
>From: Ted Bell <[log in to unmask]>
>Subject: virtual environment

------------------
Dear LLTI,

I'm sorry that I haven't replied sooner, I've been out of town for a week.

Several people were interested in more information and a few had questions and comments that I will paraphrase.  Apologies if I haven't captured the original intent of the questions.

1) Tell me more.
        a) How complex is the proposed software?
        b) How is the environment structured?
	
2) How exactly will the environment be used for language teaching?
3) Isn't it better for students to learn from real people?

Okay here goes:

1) a)  How complex is the software going to be?:
 I see the virtual environment as a natural application of the type of environment used in modern computer games.  The software components that I wish to incorporate in the environment (large scale 3d world, limited speech recognition, multi-user internet access) already exist.  Think of games like "The Sims", "Myst", or "Quake".  All of the computer worlds used in these games have features that could be applied to a language learning program.  If you are not familiar with any of these games, I apologize.  I think that the only way to appreciate them is to see them.  "The Sims" is a model of a household furnished with objects and people that are going about their lives while a godlike human "player" remodels their home.  "Myst" is a lavishly represented 3D world in which a player travels and solves puzzles by manipulating objects on the screen.  "Quake" is a fast-paced wargame where players rush through a 3D mazelike environment and attempt to wipe each other out.

In my proposed virtual environment, I'd like to have a level of detail similar to "The Sims", a level of object functionality similar or better than that of "Myst", and a scale similar to that of "Quake".  Again if these references mean nothing to you I apologize.  

I have seen attempts at a virtual environment for language learning before, but they have never had the level of detail that I'm proposing and they have not been developed at a sufficient scale.   By scale, I don't mean expense, or even graphical detail.  I just mean getting beyond a one-room/scene environment.  

The environment itself need not be language specific, and if it is sufficiently editable, it need not be culturally specific either.

b) How is the environment structured?  

Initially, the environment I have in mind will consist of a series of connected scenes in an urban setting.  Say a small house/apartment in a city with various locales to visit.  Additional connected scenes can be developed on the spot using an editor to place objects in a new setting, or specialized scenarios can be developed for technical training.  I chose an urban environment, as that will serve the needs of the most language learners for the greatest number of languages.  However there is no reason to limit scenarios to the city.

2) Here are some of the ways I propose the environment be used for language learning:

a) By solitary users:

The learner will see a representation of him/herself in the environment and navigate through the world using a keyboard, mouse, and voice commands.
A library of tasks or missions in the environment arranged by topic, context, or grammatical focus is available for the learner to work through.

Tasks, and missions:
  Learners will see a scenario or place in the environment and hear instructions accompanied by an animated demonstration.  For example, in a modern urban setting, a learner might see a kitchen, complete with fridge, table, chairs, pantry, cupboards, sink, etc.  Rank novices could be given simple instructions like opening doors, finding objects, moving things.  As a tutor, if I wanted to teach the words cupboard, plate, table, and a function like 'open'.  I might first focus attention on the various objects by centering the screen on them or highlighting them and stating their names.  I might also demonstrate the function "open" using the various things in the kitchen that open, excepting the cupboard.  All of this would incorporated in the demonstration.  Then, the learner might hear "Open the cupboard."  The learner would then have to test his/her understanding of the command by walking towards the cupboard and selecting it in some fashion (initially it will be by mouse, but for more advanced lessons it can be by verbal command).  Many objects in the world will be multifunctional, however a cupboard is pretty basic and so after selecting the cupboard, the learner can either say "open" (if the language is English and the imperative is appropriate), or choose an option from a menu with the mouse).  The cupboard will then open and reveal what may be inside it.  Further one-step-at-a-time instructions may be given so that the learner gets a plate from the cupboard and puts it on the table. 
More complicated tasks that don't involve step by step instructions, but still require listening comprehension and language use at several stages will be used as learners advance.  Eventually learners might perform a task as complicated as looking up a recipe, finding a store, conversing with a merchant, purchasing the ingredients, returning home, and assembling a peanut butter sandwich or baking a pie.  
  Contrastive feedback can be given to learners when they are stuck or seem to be going wrong.  For example if the learner is trying to open the fridge when the instructions are to "open the cupboard", a reasonable guess is that he/she is confused as to what the term "cupboard" applies and attention needs to be redirected towards the cupboard.  It only takes fairly simple AI to generate this type of feedback.

Interaction with computer generated characters:  I have yet to see good software do this.  Most software "interactions" consist of learners viewing a multimedia file of a conversation and then repeating the conversation by taking on of its parts.  This is probably more interesting than just memorizing a conversation from a text, but doesn't go very far towards having learners generate their own speech.  
I have seen text based conversations in a program called "learn French or Die" that involve some choice on the part of the user.  This allows for a flexible branching conversation, but is still fairly passive.  
I believe that simple conversations that are highly constrained by context are possible with a bit of creative programming.  Using something like the 'schema' concept from cognitive psychology, I have been able to map out highly constrained conversations.  For instance it is possible to have a flexible "conversation" with a computer generated "grocer".  To be sure, it might be a bit dull, but it's not too hard to deal with transactions such as asking where things are, determining prices, quantities, greeting, taking leave, purchasing, getting change etc.  A parser for the program doesn't need to be highly complex in order to recognize a great deal of variation in input on the part of the learner either.  If the context is highly constrained, something close to brute force algorithms actual do ok.  The nice thing about programming a schema-based conversation is that the learner doesn't need to come up with canned responses and can proceed in a more natural manner.  All that being said, the interaction with computer characters is probably the weakest aspect of the solitary use version of the environment. 

Exploration:  The environment can function as a visual dictionary where pronunciation, and meaning/function of every object on screen is available at a command.  Imagine being able to walk in to a room lay your hands on something and know how to say it, and how to describe its various functions.

Puzzles/Games:  In addition to the library of lessons, simple games with a linguistic focus can be developed in the environment.  This is probably more appropriate for children, whereas I'm actually most interested in adult learners.


b) By students and teachers in a course:
I hope that a large library of lessons can be developed for the environment. Such that instructors can select tasks/missions in the environment that are most appropriate for his/her class.  Also, the environment should be designed so that teachers or tutors can easily create new scenarios, or new lessons using existing scenarios.
Also, the networked version of the environment should prove very useful.

c) By networked learners and tutors
This is perhaps the most useful aspect of the environment for teaching speaking fluency.  A networked tutor and learner can meet online and have lessons using voice-chat.  The reality of language learning in the U.S. is that it is difficult to locate native speakers available for tutoring in most languages.  The internet removes the obstacles placed by distance.  Further, any object or place in the environment can serve as the basis for lessons.  Currently available internet tutoring does not incorporate any visual context.   My program would change that.  Another wonderful thing about the internet is that it opens up collaborative possibilities.  If an instructor designs a particularly nice scenario and library of lessons, he/she could make it available to others on the internet.  

In addition to learners and tutors meeting online for lessons, learners could just meet to chat in a language rich environment where information about vocabulary, grammar, and usage are available instantly and likely for 24 hours a day.

 
 3) The environment I envision is not designed to replace real-world language learning.  However, in a typical U.S. classroom situation in a high-school or university, students do not get many opportunities to directly engage in speaking or in listening and acting.  Verbal fluency is a skill, so is listening comprehension, they are rarely learned passively.  The constraints of class time and poor teacher student ratios mean that learners don't get enough practice in context. Consequently they can spend 4 years taking classes and still have poor speaking and listening comprehension skills. The virtual environment would provide relevant practice in a convenient and engaging way.   By the way, if I had my druthers, "conversation" classes would be taught quite a bit differently than they are typically. 


I could go on and on about various aspects of the environment.  How it can be used, what features might be nice to incorporate, etc.  I see it as more of a tool than as a simple language learning package.  I welcome any comments or questions you may have.  However, what I need most is a collaborator.  I'm not in a position to write a grant, and I have not successfully wooed any computing companies.  I have the option of postponing this project until I have a Ph.D. and can apply for funding directly, but it is likely that someone will have attempted something similar by the time I've finished.  (If someone creates a well thought out version before me, I'd be ecstatic, Iím more worried about a commercial company producing a shoddy product, but tying up the rights in patents and copyrights).

T. Bell


Find the best deals on the web at AltaVista Shopping!
http://www.shopping.altavista.com