Emergence of an Adaptive Movement from a Subjective Inference about Resource Distribution

Background: In animal foraging, the optimal search strategy in an unknown environment varies according to the context. When food is distributed sparsely and randomly, super-diffusive walks outperform normal-diffusive walks. However, super-diffusive walks are no longer advantageous when random walkers forage in a resource-rich environment. It is not currently clear whether a relationship exists between an agent’s use of local information to make subjective inferences about global food distribution and an optimal random walk strategy. Methods: Therefore, I investigated how exible exploration is achieved if an agent alters its directional rule based on local resource distribution. In the proposed model, the agent, a Brownian-like walker, estimates global resource distribution using local resource patterns and makes a decision by altering its rules. Results: I showed that the agent behaved like a non-Brownian walker and the model adaptively switched between diffusive properties depending on the resource density. This led to a more effective resource-searching performance compared with that of a simple random-walk model. Conclusion: These results demonstrate a process of optimal searching dependent on context. a which an of a The is able to the current local resource pattern local provides an for the agent to estimate sparse them.

xed length l at each time point. In contrast, the Lévy walk is a process whereby an agent takes steps of length l at each time point and where the probability density function of these steps decays asymptotically as a power law: P(l) ~ l −µ , where 1 < µ ≦ 3.
Studies of animal foraging strategies have reported that Lévy walks are e cient where resources are sparse and randomly distributed; however, they are no longer advantageous in high-density environments where resources are abundant (Bartumeus et al. 2005;Humphries and Sims 2014). For random walkers, Lévy and Brownian walks show similar exploration e ciencies if resources are extremely abundant.
Empirical studies have revealed that several animals including insects, sh, seabirds, and primates (including humans) utilize Lévy-like search patterns (Brown et al. 2007;Gonzalez et al. 2008;Raichlen et al. 2014;Ramos-Fernandez et al. 2004;Reynolds and Frye 2007;Rhee et al. 2008;Sims et al. 2008). Lévy walk patterns in primates and humans are likely that these arise through the fractal distribution of resources, the locations of which are known to the foragers. Interestingly, animals appear to select their movement strategy according to environmental conditions: they adopt Lévy movements in low-density environments and Brownian movements in high-density environments (in which searching is less necessary) (Humphries et al. 2016;López-López et al. 2013;Sims et al. 2012).
Some studies have developed random walk models and tackled optimal foraging problems related to movement-pattern formation in which the forager switches its movement strategy because of the resource items it encounters (Benhamou 2014;Benhamou and Collet 2015;Fagan 2017;Reynolds 2012). In addition, agent-based models based on pattern orientation have also been developed (Grimm and Railsback 2012); in such models, the forager is oriented based on selective pattern information, e.g., spatial structure. Thus, scanning the local spatial pattern of resource distribution can impact the forager's movement strategy. This is relevant to animals, which must guide themselves adequately using such information and be able to estimate global resource distribution when their spatial cognitive capacity is not high. However, assessing this information demands the examination of multiple patterns observed at different scales of organization or, to some extent, the use of nonlocal information (Grimm and Railsback 2012;Reynolds 2012). Consequently, the past experiences of the forager must be considered while it scans the local spatial resource distribution in order to estimate the global resource distribution. Indeed, the different past experiences of animals appear to trigger differences in decision making and behaviors, even if the animals are in similar situations (Detrain and Deneubourg 2006). In the present study, we propose a model in which an agent infers the likely direction of a nearby sparse area from their perception of neighboring cells. The agent is able to compare the current local resource pattern with the previous local resource pattern; this provides an opportunity for the agent to estimate whether an abundant or sparse area is around them. Consequently, the agent sometimes makes an incorrect inference about global resource distribution. I show that the agent in the proposed model demonstrates different random walks, which are dependent on the food resource density. The agent therefore adaptively switches its movement strategy using limited local information.

Materials And Methods
Movement and decision-making of the agent: I assumed that a single agent moves in 2D square lattices and used a Von Neumann neighborhood for all agent behaviors (movement, search, and consumption). In this study, I considered a single agent who scans the local pattern of food distribution. The agent subjectively estimates the global food distribution by comparing the current local pattern with the one before and thereby coordinates its directional rules to achieve an adaptive resource consumption. I developed two different models: a direction-modi ed model (DM model) and a Brownian-walker model (Brownian model). In each model, the agent at the coordinate (x, y) always updates its position with one of the four coordinates (x − 1, y), (x + 1, y), (x, y − 1), and (x, y + 1) at each time-step. The Brownian walker model, in which the agent always chooses one direction from the four directions -x, +x, -y, +y, is used to evaluate the resource search ability of the DM model.

Space And Environments:
Each trial was run for 1000 time-steps. The eld size was de ned as 1000 × 1000 cells and periodic boundaries were assumed. Food resources were distributed in the eld: they were randomly distributed or patchily distributed. In the latter condition, a landscape containing patchily distributed resources was used. For simplicity, it was assumed that each patch had the same size (5 × 5 cells) and contained 10 food items; the center of each patch was randomly distributed in the eld and the 10 food items were randomly distributed in each patch (patches had a uniform distribution) ( Figure S1). In addition, two patches could overlap, but patches were not permitted to roll over the periodic edges of the eld. Thus, patches with a higher density than the eld density of the randomly distributed landscape were placed at random within the eld. According to previous research, optimal random walks can differ in destructive and non-destructive scenarios (Bartumeus et al. 2005;Humphries and Sims 2014). Here, a foraging scenario was adopted in which food resources are depleted for a period after the agent nds and consumes the resource, but the resources are ultimately replenished because, in reality, they usually do not disappear permanently (Chupeau et al. 2016;Evans 1972). Furthermore, the agent consumes food items when they are located within the agent's consumption eld (≤ C onsumption radii for the agent). After the agent consumes food items, those items disappear for a period (threshold_vanish); therefore, the agent is unable to detect and consume food items that they have consumed within the threshold_vanish time-steps. After the threshold_vanish time-steps have passed, the agent can again detect and consume previously detected items when they are within the agent's consumption eld. First, I used a randomly distributed resource condition for our analysis. Subsequently, I used a patchily distributed resource condition to ensure that my proposed model was applicable to different resource environments.
Initial Conditions: I initially set the agent at the coordinates (0, 0). Note that the agent in each model begins its journey as a Brownian-like walker in each trial.

Model Description:
In the following section, the DM model is explained (it is also described using mathematical formulae in the Supplementary Material).

Direction-modi ed model
In the DM model, it is assumed that the agent cannot visually calculate global patterns of food distribution. Thus, at each time-step, the agent strictly calculates local patterns of food distribution, which indicate the spatial distribution of the food. At each time-step, the agent scans the nearest four cells for the pattern calculation. If one or three of the nearest four cells are occupied by food sources, the agent regards the local food pattern as a biased pattern to one direction; if zero or two of the nearest four cells are occupied by food sources, the agent regards the local food pattern as a non-biased pattern. Biased patterns can be "right," "left," "upper," and "lower," and they indicate the direction of food resources (see Fig. 1). For the agent, biased local food distributions represent a boundary between a (subjective) local abundant area and a (subjective) local sparse area. For example, an agent located at the coordinate (x, y) regards the local food pattern as "left" if the coordinate (x − 1, y) is occupied by food resources while the other three cells, i.e., (x + 1, y), (x, y + 1), and (x, y − 1), are empty. Similarly, an agent located at the coordinate (x, y) regards the local food pattern as "left" if the coordinate (x + 1, y) is an empty cell while the other three cells, i.e., (x − 1, y), (x, y + 1), and (x, y − 1), are occupied by food resources. Once the agent regards the local food pattern as a biased pattern, it moves a little using the current directional rule and then veri es whether or not it has nished searching the local area. Thus, a rule change occurs at the subsequent time-step.
The agent tends to move away from the local area if one cell or no cells of the nearest four cells, at time t, are occupied by food resources after it has determined the local food pattern to be biased to one direction at time t − 1. Speci cally, the agent subjectively con rms that it has nished searching the local area and decides to move away from food resources because it cannot detect abundant food resources nearby. In this case, the agent tends to move in the direction of a sparse area with a high probability and modi es the directional rule as follows: it moves in the opposite direction of the food resources with Prob high , which was calculated at the previous time-step. By doing so, the agent can move away from local food areas to search other areas. In contrast, the agent is equally likely to move in one of the other three directions with (1 -Prob high )/3.
On the other hand, the agent tends to stay within the local food area two or more cells of the nearest four cells, at time t, are occupied by food resources after it has determined the local food pattern to be biased to one direction at time t − 1. Indeed, the agent subjectively con rms that it has not yet nished searching the local area and decides to stay within that area. In this case, the agent tends to move in the direction of a sparse area with a low probability and modi es the directional rule as follows: it moves in the opposite direction of the food resources with Prob low , which was calculated at the previous time-step. As previously described, the agent is then equally likely to move in one of the other three directions with (1-Prob low )/3. Thus, the coordination of the directional rule automatically occurs based on the local food distribution if the agent determines the local food pattern to be biased at the previous time-step. Note, however, that the replacement of the directional rule with a new rule using these two-consecutive time-step events occurs only when the agent obeys the default rule, i.e., the Brownian walk. After the replacement of the rule, the agent obeys the new rule until the directional rule is returned to the default rule; this occurs when the agent reaches a local abundant area. In other words, the agent resets the directional rule at time t' when almost all of the nearest four cells (= three or four cells) are occupied by food resources at that time-step. Therefore, the agent will sometimes continue to move in a speci c direction even if it does not detect any food resources for a period. Consequently, the agent sometimes ignores Brownian-like walks and instead produces biased movements by replacing the directional rule (Fig. 2).

Parameters:
Parameters for the model are shown in Table 1. Later, the impact of changing the parameters is discussed.

Results
First, a landscape containing randomly distributed food resources was tested. Figure 3 illustrates an example of an agent's trajectory in the DM model. As shown, the agent appears to produce straight movements. The diffusiveness of search walks is a feature of interest when analyzing search e ciency.
In random walk analysis, the mean-squared distance and time-step are related as follows : Here, parameter H is determined depending on the model (H > 0.5 for a Lévy walk (super-diffusion); H = 0.5 for a Brownian walk (diffusion)). Figure 4 shows the mean-squared displacement and time-step obtained from the DM model (the mean-squared displacement (r 2 ) was obtained every 50 time-steps).
The t for parameter H according to the DM model was H ≈ 0.72, indicating that super-diffusion was achieved (R 2 = 0.99). Fig. S2 represents the relationship between step length and its cumulative frequency in each direction; power-law tailed distributions were achieved across some ranges in both directions (Fig.  S2A, x-direction: n = 42, µ = 1.72, weight of power law against exponential law = 1.00, Fig. S2B, y-direction: n = 66, µ = 2.18, weight of power law against exponential law = 1.00). Here, step lengths were de ned as distances in each direction between two consecutive 180° turns along each direction and the maximumlikelihood estimate of µ or λ was calculated, which is considered to produce the most accurate results (Raposo et al. 2003). Note that these data were obtained from one trial.
For further analysis, the e ciency of food exploitation was investigated using the DM model; its food exploitation results were compared with those of the Brownian model. The average number of consumed (detected) food items per step were calculated in each model, and the DM model performed better than the Brownian model ( Fig. 5: DM model = 0.59 ± 0.17 vs. Brownian model = 0.37 ± 0.033, χ 2 = 52.82 (proportion test), df = 1, p < 0.001). This is perhaps explained by the super-diffusive movements of the agent in DM model (Fig. 3), which can result in optimal searching in an environment with a low density of resources.
In order to investigate the in uence of resource density on the performance of the DM model, resource density was changed from 0.30 to one of 0.40, 0.50, or 0.60. As shown in Fig. 5, which illustrates the relationship between H (the slope of the mean-squared displacement) and the resource density for both models, the diffusion property of the DM model becomes almost normal-diffusive as resource density increases. Interestingly, however, the DM model also performs better than the Brownian model in terms of food exploitation, even after replacement of resource density (resource density = 0.40: DM model = 0.83 ± 0.089 vs. Brownian model = 0.49 ± 0.040, χ 2 = 92.43, df = 1, p < 0.001; resource density = 0.50: DM model = 0.97 ± 0.085 vs. Brownian model = 0.60 ± 0.051, χ 2 = 83.55, df = 1, p < 0.001; resource density = 0.60: DM model = 1.1 ± 0.097 vs. Brownian model = 0.73 ± 0.050, χ 2 = 68.45, df = 1, p < 0.001). This is perhaps explained by the agent in the DM model sometimes making straight movements after consuming food items and thereby nding new food items in a short period. Concurrently, however, the directional rule frequently resets as resource density increases, which can result in a decrease in H. Based on these overall ndings, the agent in the DM model can adaptively cope with changes in resource density.
In additional analysis, the effects of altering parameters were investigated. First, the value of parameter threshold_vanish was changed from 20 time-steps to 40 time-steps and the same analysis as described above was conducted (resource density was reset to 0.30 in these additional analyses). I found that the parameter replacement did not affect the food exploitation (DM model = 0.59 ± 0.17 vs. comparative (threshold_vanish = 40) model = 0.57 ± 0.16, χ 2 = 0.28, df = 1, p = 0.60, NS). Thus, the robustness of the DM model is maintained after the parameter threshold_vanish is changed. Other parameter effects were also investigated (Supplementary Materials).
Finally, the search e ciency for food exploitation was assessed in the DM model using a patchily distributed resource condition. For this, the threshold_vanish was set to 20 and various resource densities were used. Figure 6 shows the relationship between H and resource density observed for the DM and Brownian models. The diffusion property of the DM model almost became normal-diffusive as resource density increased; however, similar to results in a randomly distributed environment, the DM model still performed better than the Brownian model (

Discussion
In my model, the agent coordinates its directional rules by inferring the likely direction of a nearby sparse area using only the local pattern of resource distribution. Indeed, subjective sparse or abundant areas can be estimated based on the biased pattern of local resource distribution. Once an inference has been made, the agent coordinates its directional rules to stay within abundant areas or move to sparse areas by comparing temporally, consecutive local resource patterns. The search walks produced by agents in the DM model are more effective than those of Brownian walkers not only in a randomly distributed resource landscape but also in one that is patchily distributed.
According to previous simulation research, Lévy walks outperform normal-diffusive (Brownian) walks if food is sparse and randomly distributed (Bartumeus et al. 2005;Humphries and Sims 2014). On the other hand, Lévy walkers and Brownian walkers have similar exploration e ciencies if food is abundant (Bartumeus et al. 2005;Humphries and Sims 2014). Therefore, the present results are somewhat consistent with those of previous studies: the agent in the DM model performs super-diffusive walks in a low-density environment but normal-diffusive walks emerge if resource density is relatively high. In the present study, food was depleted for a period after the agent consumed it, but that food was replenished and introduced into the eld eventually. Super-diffusive walkers have an advantage in conditions where food is sparse and temporarily unavailable because they can avoid unnecessarily returning to recently visited sites. Thus, the agent spontaneously switches its movement strategy according to changes in the environment.
Several previously-developed agent-based models have focused on local spatial structure to solve optimal foraging problems and assess movement-pattern formation (Benhamou and Collet 2015;Fagan 2017;Grimm and Railsback 2012;Reynolds 2012). However, model development has rarely focused on foragers purely using local spatial structure, such as resource distribution, and autonomously lling in the gaps between the local pattern and the (subjective) global pattern. Some biological organisms cannot use a cognitive map exibly to put perceived information into context (Detrain and Deneubourg 2006;Avarguès-Weber et al. 2012); the proposed DM model might be useful for modeling an agent with a low memory capacity that cannot use a cognitive map. Such a living agent will be required to make decisions using local or limited information (Boyer and Walsh 2010;Janson and Byrne 2007;Olsson and Brown 2006;Raposo et al. 2003;Tabone et al. 2010;Valone 2006). In such situations, the agent will be unable to make clearly de ned decisions; thus, their decision making process will involve several possibilities. For example, the agent may have to determine in which direction they could nd abundant food items. Additional strategies, such as estimating food volume or patch size, could also be considered to produce several possibilities in their decision making process.
Overall, the agent in the present model autonomously lls the gaps between the local pattern and the (subjective) global pattern, and they consider the direction in which they should move. To some extent, this strategy produces an adequate movement strategy that depends on the resource environment.

Conclusions
Here, I presented a simple, highly-stylized random walk model of a foraging animal with one novel feature: agents are assumed to make inferences about global food distribution (i.e., sparse or dense, random or patchy) from their knowledge of local food distribution (i.e., in the four neighbor cells of the grid into which movement takes place). More speci cally, the agent in the proposed model, the DM model, seems to demonstrate adaptive search walks by adjusting to resource environments: super-diffusiveness is exhibited when the resource environment is sparse, whereas normal-diffusiveness is shown when the resource environment is rich (Bartumeus and Levin 2008;Humphries and Sims 2014;Reynolds and Frye 2007).

Declarations
-Ethical Approval and Consent to participate: Not applicable.
-Consent for publication: Not applicable.
-Availability of supporting data: Supplementary Material File.
-Competing interests: The author declares no con ict of interest. -Funding: Not applicable -Authors' contributions: TS conceived and designed the experiments. TS developed algorithms and performed the experiments. TS analyzed the data. TS wrote the manuscript. -Acknowledgements: Not applicable