Robotic Assistance in Coordination of Patient Care

Transcription

1 Robotcs: Scence and Systems 2016 Ann Arbor, MI, USA, June 18-22, 2016 Robotc Assstance n Coordnaton of Patent Care Matthew Gombolay, X Jesse Yang, Brad Hayes, Ncole Seo, Zx Lu, Samr Wadhwana, Tana Yu, Neel Shah, Ton Golen, & Jule Shah Massachusetts Insttute of Technology, 77 Massachusetts Avenue, Cambrdge, Massachusetts Beth Israel Deaconess Medcal Center, 330 Brooklne Avenue, Boston, Massachusetts {gombolay,xyang}@csal.mt.edu,{hayesbh,ncseo,zxlu,samrw,tanayu}@mt.edu {ntshah,tgolen}@bdmc.harvard.edu, and ule a shah@csal.mt.edu Abstract We conducted a study to nvestgate trust n and dependence upon robotc decson support among nurses and doctors on a labor and delvery floor. There s evdence that suggestons provded by emboded agents engender napproprate degrees of trust and relance among humans. Ths concern s a crtcal barrer that must be addressed before feldng ntellgent hosptal servce robots that take ntatve to coordnate patent care. Our experment was conducted wth nurses and physcans, and evaluated the subects levels of trust n and dependence on hgh- and low-qualty recommendatons ssued by robotc versus computer-based decson support. The support, generated through acton-drven learnng from expert demonstraton, was shown to produce hgh-qualty recommendatons that were accepted by nurses and physcans at a complance rate of 90%. Rates of Type I and Type II errors were comparable between robotc and computer-based decson support. Furthermore, embodment appeared to beneft performance, as ndcated by a hgher degree of approprate dependence after the qualty of recommendatons changed over the course of the experment. These results support the noton that a robotc assstant may be able to safely and effectvely assst n patent care. Fnally, we conducted a plot demonstraton n whch a robot asssted resource nurses on a labor and delvery floor at a tertary care center. I. INTRODUCTION Servce robots are beng ncreasngly utlzed across a wde spectrum of clncal settngs. They are deployed to mprove operatonal effcency by delverng and preparng supples, materals and medcatons [6, 15, 20, 35, 38]. The systems exhbt robust, autonomous capabltes for navgatng from pont to pont whle avodng obstacles [33, 34], and ntal concerns regardng physcal safety around people have largely been addressed. However, these robots are not yet wellntegrated nto the healthcare delvery process they do not operate wth an understandng of patent status and needs, and must be explctly tasked and scheduled. Ths can mpose a substantal burden upon the nurse n charge of resource allocaton, or the resource nurse, partcularly wthn fastpaced hosptal departments, such as the emergency or labor and delvery unts. Resource nurses are essentally solvng an NP-hard [5] problem on-the-fly: They assgn resources such as beds (e.g. for trage, n-patent, recovery and operatng rooms) whle subect to upper- and lower-bound temporal constrants on Ths work was supported by the Natonal Scence Foundaton Graduate Research Fellowshp Program under grant number , CRICO Harvard Rsk Management Foundaton, and Aldebaran Robotcs Inc. avalablty and consderng stochastcty n the tmng of patent progresson from one bed type to another. They must also par patents wth staff nurses, equpment and resources. The resource nurse s ob s made feasble because staff nurses understand patents statuses and needs and wll take ntatve to accomplsh some tasks wthout beng explctly drected. As the number and types of hosptal servce robots ncreases, these robots must smlarly take ntatve n order to provde a net productvty beneft. The need to explctly task many servce robots may degrade the performance of a resource nurse [9, 11, 37], whch has mplcatons for both patent safety and the well-beng of healthcare professonals [7, 24, 42, 47]. On the other hand, a robot that autonomously takes ntatve when performng tasks may make poor decsons n the absence of oversght. Furthermore, decades of research n human factors cautons aganst fully autonomous decson makng, as t contrbutes to poor human stuatonal awareness and degradaton n the human supervsor s performance [23, 41, 46, 53]. When ntegratng machnes nto human cogntve workflows, an ntermedate level of autonomy s preferred [23, 53], n whch the system provdes suggestons to be accepted or modfed by a human supervsor. Such a system would fall wthn the 4-6 range on the 10-pont scale of Sherdan s levels of automaton [41]. In ths paper, we nvestgate the human factors mplcatons of feldng hosptal servce robots that wll necessarly reason about whch tasks to perform and when to perform them. In partcular, we nvestgate trust n and dependence upon robotc decson support among nurses and doctors on a labor and delvery floor. Studes of human-automaton nteracton n avaton another safety-crtcal doman have shown that human supervsors can napproprately trust n and rely upon recommendatons made by automaton systems [16]. For example, numerous avaton ncdents have been attrbuted to human overrelance on mperfect automaton [16]. Other studes have examned the effects of changes n system relablty, and found that t led to suboptmal control allocaton strateges and reduced levels of trust n the systems [13, 14]. There s also evdence that suggestons provded by emboded agents engender over-trust and napproprate relance [44]. Ths concern s a crtcal barrer to feldng ntellgent hosptal servce robots that take ntatve to partcpate wth nurses n decson makng.

2 Fg. 1. A resource nurse must assmlate a large varety and volume of nformaton to effectvely reason about resource management for patent care. Ths paper presents three novel contrbutons to the felds of robotcs and healthcare. Frst, through human subect expermentaton wth physcans and regstered nurses, we conducted the frst known study nvolvng experts workng wth an emboded robot on a real-world, complex decson makng task comparng trust n and dependence on robotc versus computer-based decson support. Prevous studes have focused on novce users and/or smple laboratory decson tasks [4, 12, 26, 31]. Our fndngs provde the frst evdence that experts performng decson makng tasks appear to be less susceptble to the negatve effects of support embodment, as trust assessments were smlar n both the computer-based and robotc decson support condtons. Furthermore, embodment yelded performance gans compared wth computer-based support after the qualty of recommendatons changed over the course of the experment. Ths provdes encouragng evdence that ntellgent servce robots can be safely ntegrated nto the hosptal settng. Second, decson support generated through acton-drven learnng from expert demonstraton was shown to produce hgh-qualty recommendatons accepted by nurses and physcans at a complance rate of 90%. Ths ndcates that a hosptal servce robot may be able to learn context-specfc decson strateges and apply them to make reasonable suggestons for whch tasks to perform and when. Fnally, based on the prevous two fndngs, we conducted the frst test demonstraton n whch a robot asssted resource nurses on a labor and delvery floor n a tertary care center. Our robot used machne learnng computer vson technques to read the current status of the labor floor and make suggestons about resource allocaton, and used speech recognton to receve feedback from the resource nurse. To our knowledge, ths s the frst nvestgaton to feld a robotc system n a hosptal to ad n the coordnaton of resources requred for patent care. II. BACKGROUND Whle the effects of embodment on engagement n socal udgment tasks are extensvely studed and well-documented (e.g. [25, 26, 48, 49]), the relatonshp between embodment and humans levels of trust and dependence s a relatvely new area of research [4, 26, 31]. Ths topc s crucal f robots are to become more than companons, but advsors to people. Trust s defned as the atttude that an agent wll help acheve an ndvdual s goals n a stuaton characterzed by uncertanty and vulnerablty [28], and dependence s a behavoral measure ndcatng the extent to whch users accept the recommendaton of robots or vrtual agents. Measures of dependence are dstngushed accordng to whether the user makes Type I or Type II errors [17]. Type I refers to relance, or the degree to whch users accept advce from an artfcal agent when t offers low-qualty recommendatons. Type II refers to the extent to whch human users reect advce from an artfcal agent when the advce s of hgh qualty. The degrees to whch a user accepts hgh-qualty advce and reects low-qualty advce are called approprate complance and approprate relance, respectvely. Studes examnng the effects of embodment on trust and dependence necessarly nclude obectve assessments of dependence and task performance n addton to subectve assessment of the users trust n the system [4, 12, 26, 31, 40]. Scassellat et. al. [4, 31] conducted a seres of experments to compare complance rates when nteractng wth a physcally emboded robot, a vdeo of a robot and a dsemboded voce. The tasks nvolved users recevng nstructons to move obects to dfferent locatons, along wth strategy advce for solvng Sudoku-lke puzzles. The authors found that embodment was assocated wth a hgher rate of complance wth advce provded by the robot, and suggested ths ndcated a greater level of human trust for an emboded robot. Smlarly, Kesler et. al. [26] found that partcpants consumed fewer calores after recevng health advce from a physcally emboded robot, as compared wth advce from a vdeo of a robot or an on-screen anmated vrtual agent. Studes n human factors and decson support ndcate that ncreased anthropomorphsm also affects user nteractons. Pak et al. [40] evaluated how the anthropomorphc characterstcs of decson support ads assstng subects answerng questons about dabetes nfluenced subectve trust and task performance. The results ndcated that younger adults trusted the anthropomorphc decson ad more, whereas older adults were nsenstve to the effects of anthropomorphsm. Moreover, shorter queston response tme (after controllng for accuracy) was observed n both age groups, suggestng a performance gan when recevng advce from a more anthropomorphc ad. In another study, de Vsser [12] vared the degree of anthropomorphsm of a decson support system whle partcpants performed a pattern recognton task. The results ndcated that the perceved knowledgeableness of the system ncreased wth ncreasng anthropomorphsm; however, ther fndngs on dependence were nconclusve. The results from studes wth emboded robots must be nterpreted wth cauton snce they were prmarly focused on stuatons n whch robots produced relable and hghqualty recommendatons. There s a growng body of research ndcatng that the qualty of decson support cannot be reled upon, especally durng complex tasks [52]. Negatve consequences of humans blndly dependng upon mperfect emboded artfcal ntellgence have been prevously reported [44]. For example, Robnette et al. [44], conducted experments n whch a robot guded human partcpants durng a mock emergency rescue scenaro nvolvng a buldng fre. All

3 partcpants followed the robot, even when the robot led them down unsafe routes and/or dsplayed smulated malfunctons and other suspcous behavor. Such dependence upon mperfect automaton presents serous problems for robotc assstance durng safety-crtcal tasks. Ths concern s heghtened by results from studes ndcatng ncreased trust n and relance upon emboded systems as compared wth vrtual or computer-based decson support, suggestng a hgher possblty of commttng Type I errors. However, we also note that pror studes on embodment, trust and dependence were conducted wth novces rather than doman experts performng complex real-world tasks. Ths leaves us wth founded concerns, but gaps n our understandng of how human-robot nteracton mpacts the decson makng of expert resource nurses. In the next sectons, we descrbe our experment and present a postve result for servce robots n a hosptal settng, wth Type I and Type II error rates comparable to those observed for computer-based decson support. Furthermore, embodment appeared to mprove performance, as ndcated by a hgher degree of approprate complance when the qualty of advce changed md-experment. III. E XPERIMENTAL I NVESTIGATION In ths secton, we descrbe human-subect expermentaton amed at comparng trust n and dependence upon an emboded robot assstant versus computer-based decson support n a populaton of physcans and regstered nurses. The partcpants nteracted wth a hgh-fdelty smulaton of an obstetrcs department at a tertary care center. Ths smulaton provded users the opportunty to assume the roles and responsbltes of a resource nurse, whch ncluded assgnng labor nurses and scrub techncans to care for patents, as well as movng patents throughout varous care facltes wthn the department. We conducted the experment usng a wthn-subects desgn that manpulated two ndependent varables: embodment subects receved advce from ether a robot or a computer, and recommendaton qualty subects receved hgh- or lowqualty advce. Each partcpant experenced four condtons, the qualty of advce was blocked and the orderng of the condtons was counterbalanced n order to mtgate potental learnng effects. Fgure 2 depcts the expermental setup for the emboded condton. A. Hypotheses and Measures H1 Rates of approprate complance wth and relance on robotc decson support wll be comparable to or greater than those observed for computer-based decson support. Obectve measures of complance and relance were assessed based on the partcpants accept or reect response to each decson support recommendaton. Statstcs on approprate complance, approprate relance, Type I and Type II errors were recorded. H2 Robotc decson support wll be rated more favorably than computer-based decson support n terms of trust and other atttudnal measures. Numerous studes have demonstrated that emboded and anthropomorphc systems are rated Fg. 2. Experment partcpant pctured recevng advce from the robotc decson support. more favorably by users than computer-based nteractve systems. We hypotheszed that the robotc system n our study would elct ths favorable response (H2), whle engenderng approprate rates of complance and relance (H1). Ths would ndcate a postve sgnal for the successful adopton of a hosptal servce robot that partcpates n decson makng. Subectve measures of trust and atttudnal response were collected va questonnares admnstered to each partcpant after each of the four trals. Trust was assessed by a composte ratng of seven-pont Lkert-scale responses for a commonly used, valdated trust questonnare [21]. Other atttudnal questons were drawn from [29] to evaluate personalty recognton, socal responses and socal presence n human-robot nteracton, and were responded to on a 10-pont Lkert scale. B. Materals and Setup We conducted our experments usng a hgh-fdelty smulaton of a labor and delvery floor. Ths smulaton had prevously been developed through a hosptal qualty mprovement proect as a tranng tool over a year-long, rgorous desgn and teraton process that ncluded workshops wth nurses, physcans, and medcal students to ensure the tool accurately captured the role of a resource nurse. Parameters wthn the smulaton (e.g. arrval of patents, tmelnes on progresson through labor) were drawn from medcal textbooks and papers and modfed through alpha and beta testng to ensure that the smulaton closely mrrored the patent populaton and nurse experence at our partner hosptal. An Aldebaran Nao was employed for the emboded condton (Fgure 2). A vdeo of the Nao offerng advce to a partcpant wth speech and co-speech gestures s vewable at Partcpants receved advce through syntheszed speech under both the emboded and computer-based support condtons, usng a male voce drawn from the Mary Text-to-Speech System (MaryTTS) [45]. The advce was also dsplayed as text n an n-smulaton popup box under both condtons. The subect clcked a button n order to accept or reect the advce. These buttons were not clckable untl the narraton of the advce was complete; ths narraton took equal tme n both condtons. C. Expermental Procedure Seventeen physcans and regstered nurses partcpated n the experment (one man and sxteen women). The partcpants were recruted from the partner hosptal s obstetrcs department va emal and word-of-mouth.

4 Frst, partcpants provded consent for the experment and watched an 8-mnute tutoral vdeo descrbng the labor and delvery floor smulaton. The tutoral vdeo s vewable at Partcpants were nstructed to play the smulaton four tmes, wth each teraton lastng 10 mnutes, smulatng a total of 4 hours on the labor floor. The computer or emboded system would nterect durng the smulaton to make recommendatons on whch nurse should care for whch patent, and on patent room assgnments. Partcpants were asked to accept or reect the advce based on ther own udgment. They were not nformed whether the robotc or vrtual decson support coach was provdng hghor low-qualty advce. Fnally, after each of the four trals, partcpants were asked to rate ther subectve experence va a set of Lkert-scale questons, as descrbed n Secton III-A. IV. TOWARD DECISION SUPPORT - FORMULATION OF THE RESOURCE NURSE S DECISION-MAKING PROBLEM Ths secton provdes a formal representaton of the resource nurse s decson makng problem. Secton V descrbes how we mplemented the decson support based on ths formulaton. A resource nurse must solve a problem of task allocaton and schedule optmzaton wth stochastcty n the number and types of patents and the duraton of tasks. A task τ represents the set of steps requred to care for patent, and each τ s a gven stage of labor for that patent. Stages of labor are related by stochastc lower-bound constrants W τ, requrng the stages to progress sequentally. There,τ y x are stochastc tme constrants, D abs τ and D rel τ,τ y x, relatng the stages of labor to account for the nablty of resource nurses to perfectly control when a patent wll move from one stage of labor to the next. Arrvals of τ (.e. patents) are drawn from stochastc dstrbutons. The model consders three types of patents: scheduled cesarean patents, scheduled nducton patents and unscheduled patents. The set of W τ,τ y x, Dabs τ and D τ rel,τ are dependent upon patent type. Labor nurses are modeled as agents wth a fnte capacty to process tasks n parallel, where each subtask requres a varable amount of ths capacty. For example, a labor nurse may generally take care of a maxmum of two patents. If the nurse s carng for a patent who s fully and pushng (.e., the cervx s fully dlated and the patent s actvely tryng to push out the baby) or n the operatng room, the nurse may only care for that patent. Rooms on the labor floor (e.g., a labor room, an operatng room, etc.) are modeled as resources, whch process subtasks n seres. Agent and resource assgnments to subtasks are preemptable, meanng that the agent and resource assgned to care for any patent durng any step n the care process may be changed over the course of executng that subtask. In ths formulaton, t A a τ {0, 1} s a bnary decson varable for assgnng agent a to subtask τ for tme epoch [t, t + 1). t G s an nteger decson varable for assgnng a a τ certan porton of the effort of agent a to subtask τ for tme epoch [t, t + 1). t R r τ {0, 1} s a bnary decson varable for whether subtask τ s assgned resource r for tme epoch [t, t+1). H τ {0, 1} s a bnary decson varable for whether task τ and ts correspondng subtasks are to be completed. specfes the effort requred from any agent to work on U τ τ. s τ, f τ [0, ) are the start and fnsh tmes of τ. ( ) mn fn { t A a }, { t G a }, { t R r }, {H τ τ τ τ }, {s τ, f τ } a A t A a τ 1 M (1 H τ ), τ (1) τ, t (2) ( ) M 2 t A a τ H τ U τ + t G a τ ( ) t a M A + H τ 2, τ τ, t (3) r R s τ y x f τ t f τ y x s τ R τ τ τ r τ ub τ t G a τ C a, a A, t (4) 1 M (1 H τ ), τ τ τ τ, t (5) t R r τ 1, r R, t (6) f τ s τ lb τ, τ τ (7) W τ,τ, τ, τ τ, W τ,τ T C (8) f τ D rel τ,τ, τ, τ τ Drel τ,τ T C (9) Dτ abs, τ τ Dτ abs T C (10) Equaton 2 enforces that each subtask τ durng each tme epoch [t, t + 1) s assgned one agent. Equaton 3 ensures that each subtask τ receves a suffcent porton of the effort of ts assgned agent a durng tme epoch [t, t + 1). Equaton 4 ensures that agent a s not oversubscrbed. Equaton 5 ensures that each subtask τ of each task τ that s to be completed (.e., H τ = 1) s assgned one resource r. Equaton 6 ensures that each resource r s assgned to only one subtask durng each epoch [t, t+1). Equaton 7 requres the duraton of subtask τ to be less than or equal to ub τ and at least lb τ unts of tme. Equaton 8 requres that τx y occurs at least W τ unts of,τ y x tme after τ. Equaton 9 requres that the duraton between the start of τ and the fnsh of τ x y s less than D rel τ,τ x. Equaton y 10 requres that τ fnshes before D abs unts of tme have τ expred snce the start of the schedule. The stochastcty of the problem arses from the uncertanty n the upper and lowerbound of the duratons (ub τ, lb τ ) of each of the steps n carng for a patent, the number and types of patents τ and the temporal constrants T C relatng the start and fnsh of each step. These varables are a functon of the resource and staff allocaton varables t R a, t A a, and τ τ patent task state Λ τ, whch ncludes nformaton on patent type (.e. presentng wth scheduled nducton, scheduled

5 cesarean secton, or acute unplanned anomaly), gestatonal age, gravda, party, membrane status, anesthesa status, cervx status, ( tme of last exam and any co-morbdtes. Formally, {ub τ, lb τ [0, 1,..., T ]}). τ τ }, τ, T C ) A. The Role of the Resource Nurse t P ({ R a t τ, A a τ, Λ τ, t The functons of a resource nurse are to assgn nurses to take care of labor patents and to assgn patents to labor beds, recovery room beds, operatng rooms, ante-partum ward beds or post-partum ward beds. The resource nurse has substantal flexblty when assgnng beds, and ther decsons wll depend upon the type of patent and the current status of the unt n queston. They must also assgn scrub techncans to assst wth surgeres n operatng rooms, and call n addtonal nurses f requred. The correspondng decson varables for staff assgnments and room/ward assgnments n the above formulaton are t A a and t R r, respectvely. τ τ The resource nurse may accelerate, delay or cancel scheduled nductons or cesarean sectons n the event that the floor s too busy. Resource nurses may also request expedted actve management of a patent n labor. The decson varables for the tmng of transtons between the varous steps n the care process are descrbed by s τ and f τ. The commtments to a patent (or that patent s procedures) are represented by H τ. The resource nurse may also reassgn roles among nurses: For example, a resource nurse may pull a trage nurse or even care for patents herself f the floor s too busy. Or, f a patent s condton s partcularly acute (e.g., the patent has severe pre-eclampsa), the resource nurse may assgn one-to-one nursng. The level of attentonal resources a patent requres and the level a nurse has avalable correspond to varables U τ t a τ and G, respectvely. The resource nurse makes hs or her decsons whle consderng current patent status Λ τ, whch s manually transcrbed on a whteboard, shown n Fgure 1. V. IMPLEMENTATION OF DECISION SUPPORT There are two fundamental challenges to provdng decson support gudance through drect soluton of the optmzaton problem depcted above. Frst, the computatonal complexty of the problem precludes producton of real-tme solutons. The computatonal complexty ( of satsfyng) constrants n Equatons 2-10 s gven by O 2 A R T 2 C a A T, where A s the number of agents, wth each agent possessng an nteger processng capacty of C a ; there are n tasks τ, each wth m subtasks; R resources; and an nteger-valued plannng horzon of T unts of tme. In practce, there are 10 nurses (agents) who can care for up to two patents at a tme (.e., C a = 2, a A), 20 dfferent rooms (resources) of varyng types, 20 patents (tasks) at any one tme and a plannng horzon of 12 hours or 720 mnutes, yeldng a worstcase complexty of , whch s computatonally ntractable. The second challenge to decson support gudance s that the precse form of the obectve functon (Equaton 1) that resource nurses optmze for s unknown. Pror work has ndcated that doman experts are adept at descrbng the features (hgh-level, contextual and task-specfc) used n ther decson makng, yet t s more dffcult for experts to descrbe how they reason about these features [10, 43]. As such, we appled a machne learnng technque to learn a set of heurstc schedulng polces from demonstratons of resource nurse decson makng. We then appled these learned polces to produce advce for the computer-based and robotc decson support systems. A. Learnng from Resource Nurses In ths secton, we present a framework for learnng (va expert demonstraton) a set of heurstcs for resource allocaton and schedulng that emulates resource nurse decson makng. For the purposes of our experment, we focused on learnng a polcy for recommendng whch nurse should care for whch patent, and for makng patent room assgnments. We demonstrate n Secton VI that ths technque produced hghqualty recommendatons, as evdenced by an overall 90% accept rate of hgh-qualty advce. We appled acton-drven learnng rather than explctly modelng a reward functon and relyng on dynamc programmng or constrant solvers. Ths latter approach [3, 27, 36, 50, 55, 56] can quckly become computatonally ntractable for problems nvolvng hundreds of tasks and tens of agents due to memory lmtatons. Approxmate dynamc programmng approaches exst that essentally reformulate the problem as regresson [27, 32], yet the amount of data requred to regress over a large state space remans challengng, and MDP-based task allocaton and schedulng solutons exst only for smple problems [1, 51, 54]. Instead, we appled an apprentceshp schedulng algorthm [19] nspred by work n webpage rankng [22, 39]. The model representaton, a graph wth nodes and drected arcs, provdes a sutable analogy for capturng the complex temporal dependences (.e., precedence, wat and deadlne constrants) relatng tasks wthn a schedulng problem. The approach uses parwse comparsons between the actons taken (e.g., schedule agent a to complete task τ at tme t) and the set of actons not taken (e.g., unscheduled tasks at tme t) to learn relevant model parameters and schedulng polces demonstrated by the tranng examples. Ths parwse approach has the key advantage that t s nonparametrc, n that the cardnalty of the nput vector s not dependent upon the number of tasks (or actons) that can be performed n any nstance. A a ) τ Consder a set of task-resource-agent (τ R a τ assgnments, denoted π q Π. Each assgnment π q has a set of assocated features, γ πq, ndcatng patent type (.e. presentng wth scheduled nducton, scheduled cesarean secton, or acute unplanned anomaly), bed type, whether or not the bed s occuped, and staff status (.e. number of patents for whch the staff member s servng as prmary nurse, coverng nurse, baby nurse, or scrub techncan). Next, consder a set of m observatons, O = {O 1, O 2,..., O m }. Each observaton conssts of a feature vector descrbng the task-resource-agent

6 tuple π q scheduled by the expert demonstrator (ncludng a null task τ, resource r or agent a f no task, resource or agent was scheduled). The goal s to then learn a polcy that correctly determnes whch task-resource-agent tuple π q to schedule as a functon of feature state. rank θ m π q,π r := [ γ πq γ πr ], y m πq,π r = 1, π r Π\π q, O m O π q scheduled n O m (11) rank θ m π r,π q := [ γ πr γ πq ], y m πr,π q = 0, π r Π\π q, O m O π q scheduled n O m (12) π q = argmax π q Π f prorty (π q, π r ) (13) π q Π In order to learn to correctly assgn the subsequent task to the approprate resource and/or agent, we transform each observaton O m nto a new set of observatons by performng parwse comparsons between the scheduled assgnment π q and the set of assgnments s that were not scheduled (Equatons 11-12). Equaton 11 creates a postve example for each observaton n whch a π q was scheduled. Ths example conssts of the nput feature vector, φ m π, and q,π r a postve label, y π m q,π r = 1. Each element of the nput feature vector φ m π q,π r s computed as the dfference between the correspondng values n the feature vectors γ πq and γ πr, descrbng scheduled assgnment π q and unscheduled task π r. Equaton 12 creates a set of negatve examples wth y π m r,π q = 0. For the nput vector, we take the dfference of the feature values between unscheduled assgnment π r and scheduled assgnment π q. We appled these observatons to tran a decson-tree classfer f prorty (π q, π r ) {0, 1} to predct whether t s better to make the task-resource-agent assgnment π q as the next assgnment rather than π r. Gven ths parwse classfer, we can determne whch sngle assgnment π q * s the hghest-prorty assgnment accordng to Equaton 13 by determnng whch assgnment s most often of hgher prorty n comparson to the other assgnments n Π. In our experments, f prorty (π q, π r ) was appled drectly to generate hgh-qualty recommendatons. We generated lowqualty advce usng two methods: The frst method recommended the acton that mnmzed Equaton 13, nstead of maxmzng t. Ths approach would typcally generate nfeasble advce (e.g., move a patent to a room that s currently occuped). A second method was appled to offer low-qualty but feasble advce (e.g., assgn a post-operatng patent to trage). Ths was acheved by evaluatng Equaton 13 after flterng the space of possble actons to nclude only feasble actons (per the constrants n Equatons 2-10). Recommendatons for the low-qualty condton were produced by randomly selectng between these two methods n order to mtgate orderng effects. The dataset used for tranng was generated by seven resource nurses workng wth the smulaton for a total of 2 1 /2 TABLE I CONFUSION MATRIX FOR PARTICIPANTS SHOWN AS A RAW COUNT AND PERCENTAGE OF THE WHOLE. Response Robotc Decson Support Accept Reect Hgh 130 (44.5%) 16 (5.48%) Advce Qualty Low 16 (5.48%) 130 (44.5%) Response Vrtual Decson Support Accept Reect Hgh 134 (45.3%) 14 (4.78%) Advce Qualty Low 19 (6.48%) 126 (43.0%) TABLE II CORRECT ACCEPT AND REJECT DECISIONS MADE WITH COMPUTER-BASED (C-ACCEPT, C-REJECT) VERSUS ROBOTIC (R-ACCEPT, R-REJECT) DECISION SUPPORT, AS A FUNCTION OF TRIAL NUMBER, SHOWN AS A RAW COUNT AND PERCENTAGE OF THE WHOLE. Tral Number Bad Advce Good Advce C-Accept 5 (10.4%) 4 (6.7%) 41 (82.0%) 49 (92.5%) R-Accept 9 (17.6%) 5 (9.6%) 43 (91.5%) 44 (93.6%) Tral Number Good Advce Bad Advce C-Reect 2 (28.6%) 1 (2.8%) 11 (73.3%) 20 (87.0%) R-Reect 3 (8.6%) 1 (10.0%) 21 (84.0%) 16 (94.1%) hours, smulatng 60 hours of elapsed tme on a real labor floor. Ths yelded a dataset of more than 3, 013 ndvdual decsons. None of the seven resource nurses who contrbuted to the dataset partcpated n the experment. VI. RESULTS We report statstcal testng of our hypotheses here. We defned statstcal sgnfcance at the α = 0.05 level. A. Analyss & Dscusson of H1 Obectve measures of complance and relance were assessed based on the partcpant s accept or reect responses to each decson support recommendaton. Statstcs on hts, msses, false alarms and correct reectons are shown n Table I. Results from a z-test for two proportons ndcated no statstcally sgnfcant dfference n the Type II error rates for the robotc (p R = 13.1%) and computer-based (p C = 11.0%) decson support condtons (z = 0.562, p = 0.713), nor n the rates of correct accept responses to hgh-qualty advce (p R = 90.5%, p C = 89.0%, p = 0.713) and reect responses to low-qualty advce (p R = 86.9%, p C = 89.0%, p = 0.287) across the two condtons. Results from a TOST equvalence test usng two z-tests for two proportons ndcated that the rates of error, approprate complance and approprate relance between the robotc and vrtual decson support condtons were equvalent wthn 95% confdence. We also analyzed the rates of Type I and Type II errors n the second and thrd trals, at the transton n advce qualty (Table II). Fsher s exact test found a sgnfcant dfference

7 TABLE III SUBJECTIVE MEASURES POST-TRIAL QUESTIONNAIRE WITH STATISTICAL SIGNIFICANCE. QUESTIONS 1-5 WERE RESPONDED TO ON A 7-POINT SCALE, AND QUESTIONS 6-9 ON A 10-POINT SCALE. Trust and Embodment n Human-Robot Interacton 1. I am suspcous of the system s ntent, actons or outputs. 2. I thnk I could have a good tme wth ths decson support coach. 3. People wll fnd t nterestng to use ths decson support coach. 4. Whle you were nteractng wth ths decson-support coach, how much dd you feel as f t were an ntellgent beng? 5. Whle you were nteractng wth ths decson-support coach, how much dd you feel as f t were a socal beng? 6. Unsocable/Socable. 7. Machne-Lke/Lfe-Lke. n the rate of ncorrect accept of low qualty advce (Type I error) across the second and thrd trals for the computer-based decson support (6.7% vs. 26.7%, p = 0.046), but not for the robotc support (9.6% vs. 16.0%, p = 0.461). A sgnfcant dfference was also found n the rate of ncorrect reect of hgh-qualty advce (Type II error) across the second and thrd trals for the computer-based decson support (2.8% vs. 18.0%, p = 0.040), but not for robotc decson support (10.0% vs. 8.5%, p 1.0). In other words, partcpants rate of Type I error assocated wth computer-based support ncreased sgnfcantly when partcpants had receved hghqualty advce n the prevous tral. Smlarly, the rate of Type II error assocated wth computer-based support ncreased sgnfcantly when partcpants had receved low-qualty advce n the prevous tral. No such sgnfcant dfferences were found for the robotc support condtons. H1 Takeaway: These results support H1, n that Type I and Type II error rates were comparable between robotc and computer-based decson support. Furthermore, embodment appeared to offer performance gans, as ndcated by lower error rates after the qualty of recommendaton changed mdexperment. These are encouragng fndngs because they provde evdence that a robotc assstant may be able to partcpate n decson makng wth nurses wthout elctng napproprate dependence. One potental ratonale for these results s that experts may be less susceptble to the negatve effects of embodment, as has been documented for experenced users nteractng wth anthropomorphc agents [40]. We note that our study was conducted wth a statonary robot, n whch movement was lmted to co-speech gestures. Further nvestgaton s warranted for stuatons n whch experts nteract wth moble servce robots that partcpate n decson-makng. B. Analyss & Dscusson of H2 A composte measure of trust was computed, as n [21]. Results from a repeated-measures ANOVA (RANOVA) demonstrated a statstcally sgnfcant ncrease n the average ratng for the decson support system under the hgh-qualty advce condton (M = 5.39, SD = 0.666) as compared wth the low-qualty condton (M = 3.49, SD = 1.26) (F (1, 14) = 46.3, p < 0.001). However, a RANOVA yelded no statstcally sgnfcant dfference n trust between the robotc (M = 4.41, SD = 1.32) and computer-based (M = 4.48, SD = 1.47) embodment condtons (F (1, 14) = 0.450, p = 0.513). Results from a TOST equvalence test, usng two t-tests, ndcated that subects trust ratngs for the computer-based and robotc support were wthn one pont of one another on a 7-pont Lkert Scale. We observed sgnfcant dfferences n the atttudnal assessment of the robotc versus computer-based decson support condtons for Questons 2, 3, 5, 6 n Table III, ndcatng that partcpants rated the robotc system more favorably. The result was establshed usng a two-way omnbus Fredman test, followed by parwse Fredman tests. The test statstcs for the parwse Fredman tests were p = 0.028, 0.007, 0.043, and 0.005, respectvely. Strkngly, there was not a sngle queston (out of 37) for whch partcpants rated the computer-based decson support sgnfcantly better than the robotc support. We also found that the subectve percepton of the character of the robot was sgnfcantly less senstve to transtons n advce qualty than the computer-based decson support. We computed the frequency wth whch the ratngs of one embodment condton subsumed the other, and vce versa. Specfcally, we defned x R,L as the Lkertscale ratng for a gven queston and a partcular partcpant n the robotc low-qualty advce condton, and lkewse for the hgh-qualty condton, x R,H. The varables x C,L, x C,H were smlarly defned for the computer-based lowand hgh-qualty condtons. The robotc condton was defned as subsumng the computer-based condton f ether mn(x R,L, x R,H ) mn(x C,L, x C,H ) max(x C,L, x C,H ) < max(x R,L, x R,H ) or mn(x R,L, x R,H ) < mn(x C,L, x C,H ) max(x C,L, x C,H ) max(x R,L, x R,H ), and vce versa for the computer-based condton subsumng the robotc condton. A χ 2 test ndcated that the partcpants subectve evaluaton accordng to Questons 1, 4, 6, 7 (p = 0.045, 0.022, and , respectvely) changed more sgnfcantly under the computer-based condton than the robotc condton. There were no questons for whch the response changed more sgnfcantly under the robotc condton versus the computerbased condton. In other words, the subectve assessment of the robot was more robust to advce qualty changes than the computer-based decson support. Further nvestgaton s warranted to determne whether these effects persst over tme as the users habtuate to nteracton wth the robot. H2 Takeaway: Our fndngs support H2 n that the robotc system was rated more favorably on atttudnal assessment than computer-based decson support, even as t engendered approprate dependence. It s nevtable that a servce robot wll occasonally make poor-qualty suggestons, and we postvely note that the robot engendered greater tolerance of errors than the computer-based decson support. These results ndcate a postve sgnal for successful adopton of a robot that partcpates n a resource nurse s decson makng. VII. PILOT DEMONSTRATION OF A ROBOTIC ASSISTANT ON THE LABOR AND DELIVERY FLOOR Based on the postve results of our experment, we conducted a plot demonstraton n whch a robot asssted resource

8 Fg. 3. Images of the robot system n acton on the labor floor. nurses on a labor and delvery floor at a tertary care center. A. Robot System Archtecture The system was comprsed of subsystems provdng the vson, communcaton and decson support capabltes. Vson System: In our experments, the statuses of patents, nurses and beds were provded and updated n the smulaton. In contrast, nurses and robots on a real labor floor must read handwrtten nformaton off of a whteboard (.e., dashboard ) depcted n Fgure 1. Extractng and parsng ths nformaton autonomously wth hgh accuracy and relablty presents a substantal techncal challenge. We make two assumptons to address ths: (1) that the set of physcan and nurse names s closed and known n advance, and (2) that patent names are transcrbed for the robot upon patent arrval. In our demonstraton, we leveraged the structured nature of the dashboard to ntroduce prors that ensured patent nformaton was nterpretable. Rows on the dashboard ndcate room assgnments, whle columns ndcate patent parameters (e.g., attendng physcan, gestatonal age, etc.). Once our robot captured an mage of the dashboard on the labor and delvery floor, we appled a Canny edge detecton operator [8] and Hough transformaton [18] to solate the handwrtng n ndvdual grd cells. The contents of each grd cell were processed usng a classfcaton technque approprate to the data type theren. Numerc felds were parsed usng a Convolutonal Neural Network (CNN)1 traned on MNIST data, whle alphabetcal felds wth known sets of possble values (e.g. attendng physcan, nurse names) were parsed usng a mult-class CNN traned on handwrtng2. Handwrtng samples (28 unquely wrtten alphabets) were used as a bass for generatng classfer tranng data. Fonts were created from the provded samples and used (along wth system fonts) to create a large set of bnary mages contanng samples of nurse names. These synthetc wrtng samples were constructed wth a range of appled translatons, scalngs, and kernng values wthn a 75x30 pxel area. The vson system was used to determne the current status of patent-nurse allocatons, nurse role nformaton and room usage. Pror to deployment, we performed a valdaton of the vson system and found our recognton system to correctly classfy handwrtten samples across 15 classes (names) wth 83.7% overall accuracy and 97.8% average accuracy. These 1 Thanks to Mkhal Srontenko for developng ths package, whch s avalable at 2 We utlze a network wth a the followng archtecture: 75x30 nput layer 5x5 kernel convoluton layer 2x2 kernel maxpool layer 5x5 kernel convoluton layer 2x2 kernel maxpool layer 100 node dense layer classfcaton layer. results were obtaned wthout performng any envronmental manpulatons (adustng lghtng, usng hgh-resoluton cameras, etc.). In the plot deployment, our vson system asssted humans wth transcrpton of patent data. Communcaton: CMUSphnx [2] was employed for robot speech recognton. To acheve hgh performance n a lve settng, we defned a lst of template-based phrases a user mght utter, such as Where should I move the patent n room [#]? or Who should nurse [Name] take care of? All possble nstantatons were enumerated based on nformaton avalable a pror (e.g., the lst of nurse names). Levenshten dstance [30] was computed to nfer the phrase most lkely uttered by the speaker, and the approprate correspondng query was ssued to the decson support system. Decson Support: The lve plot demonstraton of the robot used the same mechansm for generatng decson support as that used durng our experments. However, unlke the experments, the decson support system s nput was taken from the vson subsystem, and the user query from the communcaton subsystem. The set of possble actons to be recommended was fltered accordng to the query as recognzed by the communcaton subsystem. For example, f the user asked, Where should I move the patent n room 1A?, actons that would change nurse assgnments were not consdered. The recommended acton was communcated to the user va text-to-speech software. Feedback from Nurses and Physcans: We conducted a test demonstraton on the labor floor (Fgure 3). Three users nteracted wth the robot over the course of three hours. Ten queres were posed to the robot; seven resulted n successful exchanges and three faled due to background nose. A lve recordng of the demo can be seen at After nteractng wth the robotc support, User 1, a physcan, sad I thnk the [robot] would allow for a more even dsperson of the workload amongst the nurses. In some hosptals...more unor nurses were gven the next patent...more senor nurses were allowed to only have one patent as opposed to two. User 2, a resource nurse sad, New nurses may not understand the constrants and complextes of the role, and I thnk the robot could help gve her an algorthm... that she can practce, repeat, and become famlar wth so that t becomes second nature to her. User 3, a labor nurse offered, I thnk you could use ths robot as an educatonal tool. VIII. C ONCLUSION Ths paper addresses two barrers to feldng ntellgent hosptal servce robots that take ntatve to partcpate wth nurses n decson makng. We fnd expermental evdence that experts performng decson makng tasks may be less susceptble to the negatve effects of support embodment. Further our decson support was able to produce context-specfc decson strateges and apply them to make reasonable suggestons for whch tasks to perform and when. Fnally, based on the prevous two fndngs, we conducted a frst successful test demonstraton n whch a robot asssted resource nurses on a labor and delvery floor n a tertary care center.

9 REFERENCES [1] A novel mult-agent renforcement learnng approach for ob schedulng n grd computng. Future Generaton Computer Systems, 27(5): , [2] Cmu sphnx open source speech recognton toolkt, January URL [3] Peter Abbeel and Andrew Y. Ng. Apprentceshp learnng va nverse renforcement learnng. In ICML. ACM, ISBN do: / URL [4] Wlma A Banbrdge, Justn W Hart, Elzabeth S Km, and Bran Scassellat. The benefts of nteractons wth physcally present robots over vdeo-dsplayed agents. Internatonal Journal of Socal Robotcs, 3(1):41 52, [5] D. Bertsmas and R. Wesmantel. Optmzaton over Integers. Dynamc Ideas, [6] Rchard Bloss. Moble hosptal robots cure numerous logstc needs. Industral Robot: An Internatonal Journal, 38(6): , [7] L. Brandenburg, P. Gabow, G. Steele, J. Toussant, and B. J. Tyson. Innovaton and best practces n health care schedulng. Techncal report, [8] John Canny. A computatonal approach to edge detecton. IEEE Transactons on Pattern Analyss and Machne Intellgence, (6): , [9] Jesse YC Chen, Mchael J Barnes, and Mchelle Harper- Scarn. Supervsory control of multple robots: Humanperformance ssues and user-nterface desgn. Systems, Man, and Cybernetcs, Part C: Applcatons and Revews, IEEE Transactons on, 41(4): , [10] Tsang-Hsang Cheng, Chh-Png We, and Vncent S. Tseng. Feature selecton for medcal data mnng: Comparsons of expert udgment and automatc approaches. In Proc. CBMS, pages , [11] Mary L Cummngs and Stephane Guerlan. Developng operator capacty estmates for supervsory control of autonomous vehcles. Human Factors: The Journal of the Human Factors and Ergonomcs Socety, 49(1):1 15, [12] E. J. de Vsser, F. Krueger, P. McKnght, S. Sched, M. Smth, S. Chalk, and R. Parasuraman. The world s not enough: Trust n cogntve agents. Proceedngs of the Human Factors and Ergonomcs Socety Annual Meetng, 56(1): , [13] Munal Desa, Mkhal Medvedev, Marynel Vázquez, Sean McSheehy, Sofa Gadea-Omelchenko, Chrstan Bruggeman, Aaron Stenfeld, and Holly Yanco. Effects of changng relablty on trust of robot systems. In Human-Robot Interacton (HRI), th ACM/IEEE Internatonal Conference on, pages IEEE, [14] Munal Desa, Poornma Kanarasu, Mkhal Medvedev, Aaron Stenfeld, and Holly Yanco. Impact of robot falures and feedback on real-tme trust. In Proceedngs of the 8th ACM/IEEE Internatonal Conference on Human-robot Interacton, HRI 13, pages , Pscataway, NJ, USA, IEEE Press. ISBN URL [15] Ncole DGose. Hosptals hrng robots, February URL Perpherals/Systems/Hosptals hrng robots.aspx. [16] R. K. Dsmukes, B. A. Berman, and L. D. Loukopoulous. The Lmts of Expertse: Rethnkng Plot Error and the Causes of Arlne Accdents. Ashgate Publshng, [17] Stephen R Dxon and Chrstopher D Wckens. Automaton relablty n unmanned aeral vehcle control: A relance-complance model of automaton dependence n hgh workload. Human Factors: The Journal of the Human Factors and Ergonomcs Socety, 48(3): , [18] Rchard O Duda and Peter E Hart. Use of the hough transformaton to detect lnes and curves n pctures. Communcatons of the ACM, 15(1):11 15, [19] Matthew Gombolay, Reed Jensen, Jessca Stgle, Sung- Hyun Son, and Jule Shah. Apprentceshp schedulng: Learnng to schedule from human experts. In Proceedngs of the Internatonal Jont Conference on Artfcal Intellgence (IJCAI), New York Cty, NY, U.S.A., July [20] John Hu, Aaron Edsnger, Y-Je Lm, Nck Donaldson, Maro Solano, Aaron Solochek, and Ronald Marchessault. An advanced medcal robotc system augmentng healthcare capabltes-robotc nursng assstant. In Robotcs and Automaton (ICRA), 2011 IEEE Internatonal Conference on, pages IEEE, [21] J.-Y. Jan, A. M. Bsantz, and C. G. Drury. Foundatons for an emprcally determned scale of trust n automated systems. Internatonal Journal of Cogntve Ergonomcs, 4(1):53 71, [22] Rong Jn, Hamed Valzadegan, and Hang L. Rankng refnement and ts applcaton to nformaton retreval. In Proceedngs of the 17th Internatonal Conference on World Wde Web, pages ACM, ISBN do: / URL [23] Davd B Kaber and Mca R Endsley. Out-of-the-loop performance problems and the use of ntermedate levels of automaton for mproved control system functonng and safety. Process Safety Progress, 16(3): , [24] S. M. Kehle, N. Greer, I. Rutks, and T. Wlt. Interventons to mprove veterans access to care: A systematc revew of the lterature. Journal of General Internal Medcne, 26(2): , [25] Cory D Kdd and Cyntha Breazeal. Effect of a robot on user perceptons. In Intellgent Robots and Systems, 2004.(IROS 2004). Proceedngs IEEE/RSJ Internatonal Conference on, volume 4, pages IEEE, 2004.

10 [26] Sara Kesler, Aaron Powers, Susan R Fussell, and Crsten Torrey. Anthropomorphc nteractons wth a robot and robot-lke agent. Socal Cognton, 26(2): , [27] G. Kondars, S. Osentosk, and P. Thomas. Value functon approxmaton n renforcement learnng usng the fourer bass. In Proc. AAAI, pages , [28] John D. Lee and Katrna A. See. Trust n automaton: Desgnng for approprate relance. Human Factors, 46 (1):50 80, [29] K. W. Lee, W. Peng, S.-A. Jn, and C. Yan. Can robots manfest personalty?: An emprcal test of personalty recognton, socal responses, and socal presence n humanrobot nteracton. Journal of Communcaton, 56 (4): , [30] Vladmr I Levenshten. Bnary codes capable of correctng deletons, nsertons, and reversals. In Sovet physcs doklady, volume 10, pages , [31] Danel Leyzberg, Samuel Spauldng, and Bran Scassellat. Personalzng robot tutors to ndvduals learnng dfferences. In Proceedngs of the 2014 ACM/IEEE nternatonal conference on Human-robot nteracton, pages ACM, [32] V. Mnh, K. Kavukcuoglu, D. Slver, A. A. Rusu, J. Veness, M G. Bellemare, A. Graves, M. Redmller, A. K. Fdeland, G. Ostrovsk, S. Petersen, C. Beatte, A. Sadk, I. Antonoglou, H. Kng, D. Kumaran, D. Werstra, S. Legg, and D. Hassabs. Human-level control through deep renforcement learnng. Nature, 518(7540): , [33] Ryota Mura, Tadash Saka, Yuma Honda, et al. Recognton of 3d dynamc envronments for moble robot by selectve memory ntake and release of data from 2d sensors. In System Integraton (SII), 2012 IEEE/SICE Internatonal Symposum on, pages IEEE, [34] Ryota Mura, Tadash Saka, Hroyuk Kawano, Yoshhko Matsukawa, Yuma Honda, KC Campbell, et al. A novel vsble lght communcaton system for enhanced control of autonomous delvery robots n a hosptal. In System Integraton (SII), 2012 IEEE/SICE Internatonal Symposum on, pages IEEE, [35] Blge Mutlu and Jod Forlzz. Robots n organzatons: the role of workflow, socal, and envronmental factors n human-robot nteracton. In Human-Robot Interacton (HRI), rd ACM/IEEE Internatonal Conference on, pages IEEE, [36] P. Odom and S. Nataraan. Actve advce seekng for nverse renforcement learnng. In Proceedngs of AAAI, pages , [37] Dan R Olsen Jr and Stephen Bart Wood. Fan-out: measurng human control of multple robots. In Proceedngs of the SIGCHI conference on Human factors n computng systems, pages ACM, [38] Al Gürcan Özkl, Zhun Fan, Steen Dawds, H Aanes, Jens Klæstrup Krstensen, and Km Hardam Chrstensen. Servce robots for hosptals: A case study of transportaton tasks n a hosptal. In Automaton and Logstcs, ICAL 09. IEEE Internatonal Conference on, pages IEEE, [39] Tapo Pahkkala, Evgen Tsvtsvadze, Antt Arola, Jorma Boberg, and Tapo Salakosk. Learnng to rank wth parwse regularzed least-squares. In SIGIR 2007 Workshop on Learnng to Rank for Informaton Retreval, pages 27 33, [40] R. Pak, N. Fnk, M. Prce, B. Bass, and L. Sturre. Decson support ads wth anthropomorphc characterstcs nfluence trust and performance n younger and older adults. Ergonomcs, 55(9): , [41] R. Parasuraman, T.B. Sherdan, and Chrstopher D. Wckens. A model for types and levels of human nteracton wth automaton. Trans. SMC-A, 30(3): , [42] S. D. Pzer and J. C. Prentce. What are the consequences of watng for health care n the veteran populaton? Journal of General Internal Medcne, 26(2): , [43] Hema Raghavan, Omd Madan, and Rose Jones. Actve learnng wth feedback on features and nstances. Journal of Machne Learnng Research, 7: , December ISSN [44] P. Robnette, A. M. Howard, and A. R. Wagner. Overtrust of robots n emergency evacuaton scenaros. In Proc. HRI, [45] Marc Schröder and Jürgen Trouvan. The german textto-speech synthess system mary: A tool for research, development and teachng. Internatonal Journal of Speech Technology, 6(4): , [46] Thomas B Sherdan. Adaptve automaton, level of automaton, allocaton authorty, supervsory control, and adaptve control: Dstnctons and modes of adaptaton. Systems, Man and Cybernetcs, Part A: Systems and Humans, IEEE Transactons on, 41(4): , [47] S. A. Shpman and C. A. Snsky. Expandng prmary care capacty by reducng waste and mprovng effcency of care. Health Affars, 32(11): , [48] Lela Takayama and Carolne Pantofaru. Influences on proxemc behavors n human-robot nteracton. In Intellgent Robots and Systems, IROS IEEE/RSJ Internatonal Conference on, pages IEEE, [49] Adrana Tapus, Crstan Tapus, and Maa Matarć. The role of physcal embodment of a therapst robot for ndvduals wth cogntve mparments. In Robot and Human Interactve Communcaton, RO-MAN The 18th IEEE Internatonal Symposum on, pages IEEE, [50] Adam Vogel, Deepak Ramach, Rakesh Gupta, and Antone Raux. Improvng hybrd vehcle fuel effcency usng nverse renforcement learnng. In Proceedngs of AAAI, pages , [51] Y.-C. Wang and J. M. Usher. Applcaton of renforcement learnng for agent-based producton schedulng. Eng. Appl. Artf. Intell., 18(1):73 82, [52] C. D. Wckens, J. G. Hollands, S. Banbury, and R. Para-