In this study, the Zero-Shot learning capabilities of LLMs are employed to manage the multi-agent inventory management systems of Behpakhsh Company. This capability enables adaptive and informed decision-making without the need for prior training or explicitly provided examples, thereby enhancing system flexibility.

The proposed model offers a high level of explainability, further enhanced by Chain-of-Thought (CoT) reasoning. This feature facilitates a clearer understanding of the model's behavior and increases trust in its outcomes, making it a more reliable system compared to traditional heuristic and reinforcement learning approaches.

Our model dynamically adapts to different demand scenarios, thereby minimizing costs and preventing inventory shortages. Through extensive evaluation across diverse scenarios, this capability demonstrates improved efficiency in Behpakhsh Company's SCM and optimizes inventory management processes.

These contributions enable Behpakhsh Company to enhance its supply chain productivity by leveraging advanced technologies and to deliver more reliable and transparent operational performance.

A multi-layer, multi-period production–inventory system is designed to model and simulate a typical multi-stage supply chain for the production and distribution of non-perishable goods. As illustrated in Fig. 2, each stage of the supply chain consists of an inventory storage area (for storing essential materials required for production) and a production facility (for converting raw materials into products).

System characteristics:

I. The output materials of stage i are used as the input materials for stage i–1.

II. The stages are indexed in descending order, from stage M–1 (the upstream beginning of the supply chain) to stage 0, which represents the retail level.

III. Production at each stage is constrained by both the production capacity and the available inventory at that stage.

IV. The flow of raw materials starts from upstream stages and eventually reaches the retail stage to satisfy customer demand.

Time periods (T): the supply chain is divided into discrete time periods t. Each simulation starts at t = 0 with initial conditions, and during each time period T, the following events occur:

Shipment review: each stage receives its inbound shipments that arrive after completing the corresponding transportation lead times.

Order and demand review: each stage places replenishment orders to its corresponding suppliers. Orders are placed based on available production capacity and suppliers' inventory levels. Customer demand at the retail stage is fulfilled based on available inventory.

Order and demand fulfillment: each stage delivers products up to its maximum feasible capacity to satisfy incoming demands or replenishment orders. Orders and demands that cannot be fulfilled within the current time period are prioritized in the subsequent period.

Profit calculation: each stage calculates the costs and profits associated with the following activities: product sales, raw material ordering, penalties for delayed fulfillment, and Holding costs for excess inventory.

This multi-layer, multi-period production–inventory system provides the foundational structure for analyzing and simulating the supply chain processes of the Behpakhsh Company. It accurately models transportation, production, and inventory management processes. The proposed InvAgent leverages this structure to enhance supply chain performance and reduce operational costs.

In Fig. 2, Raw materials flow through each stage, which includes inventory storage and production facilities. The upstream plant at stage i supplies intermediate products to stage i-1 below, where they are stored as inventory. Stage 0 (retail) supplies finished products to meet customer demand. Considering the items mentioned in Table 1, the overall inventory management problem (IMP), which is taken from Hubbs et al. [9], can be described using the following equations:

Im,t = Im,t−1 + Rm,t−Lm − Sm,t, for all m ∈ M, (1) Rm,t = min(Bm+1,t−1 + Om,t,Cm+1, Im+t,t−1 + Rm+1,t−Lm+1) , for all m = 0, … , M − 2, (2a) RM−1,t = OM−1,t, (2b) Sm,t = Rm−1,t, for all m = 1, … , M − 1, (3a) S0,t = min(B0,t−1 + Dt,C0, I0,t−1 + R0,t−L0), (3b) Bm,t = BM,t−1 + Om−1,t − Sm,t, for all m = 1, … , M − 1, (4a) B0,t = B0,t−1 + Dt − S0,t, (4b) Pm,t = pmSm,t − rmRm,t − kmBm,t − hmIm,t, for all m ∈ M. (5)

In Eq (1), the current inventory at stage m at the end of the current time period t is equal to the ending inventory of the previous period, plus the fulfilled order from period t − Lm, minus the sales during the current period. In Eq (2.a), the fulfilled orders at stage m placed during time period t are determined by the minimum of the following three components: 1) the previous backlog at the downstream stage plus the newly requested orders, 2) the production capacity of the upstream stage, and 3) the total available inventory at the upstream stage m + 1 at the beginning of time period t, which includes the remaining inventory from the previous period and newly arrived orders after accounting for the transportation lead time. The final fulfilled order is the minimum of these three values, ensuring the order does not exceed any of them. It ensures the supply chain operates within inventory and capacity limits, preventing overcommitment and inventory shortages. Eq (2.b) indicates that requested orders at the highest stage are always fulfilled, since an unlimited supply of raw materials is assumed at this stage. As shown in Eq (3.a), sales are always equal to the fulfilled orders, except at stage 0 (the retailer). In Eq. (3.b), sales at stage 0 (the retailer) during time period t are determined by the minimum of the following three components: 1) the previous backlog at stage 0 plus the current customer demand, 2) the production capacity at stage 0, and 3) the total available inventory at stage 0 at the beginning of time period t, which consists of the remaining inventory from the previous period and newly fulfilled orders after accounting for transportation lead time. It ensures that retail sales do not exceed total demand, production capacity, or available inventory. In Eq. (4.a), the backlog at stage m during time period t, for all stages except the retailer, is calculated as the sum of the previous backlog at that stage and the orders requested from the downstream stage, minus the sales at stage m. In Eq. (4.b), the backlog at stage 0 (the retailer) during time period t is calculated similarly to Eq. (4.a); however, the requested order is replaced by customer demand, since the retailer interacts directly with customers. Finally, in Eq. (5), the profit at each stage m during time period t is calculated as sales revenue minus procurement costs, unmet demand penalty costs, and inventory holding costs.

In this project, we introduce InvAgent, a LLM–based multi-agent inventory management system designed for supply chain optimization. InvAgent consists of several key agents, including a user proxy and one agent for each stage of the supply chain. The user proxy acts as an intermediary between the environment and all supply chain agents, facilitating communication and data exchange management.

The methodological framework of InvAgent is illustrated in Fig. 1 and proceeds through the following steps:

I. The user proxy resets the environment at the beginning of the first episode.

II. The user proxy requests the current episode state for each stage from the environment.

III. The user proxy provides the state of each stage to the corresponding agent and requests an action from it. IV. The user proxy sends the agents' actions to the environment and receives the next state and the reward for that step.

V. The user proxy determines whether the simulation has terminated; if not, the simulation returns to Step 2. At the beginning of the simulation, we generate system messages for the agents (as shown in Fig. 3) that provide essential information, such as definitions, roles, and objectives, within the supply chain. The agent state Sm,tS_{m,t}Sm,t and action am,ta_{m,t}am,t are defined as follows:

sm,t = [cm, pm, rm, km, hm, Lm, Im,t−1, Bm,t−1, Bm+1,t−1, Sm,t−Lmax, … , Sm,t−1, 0, … ,0, Rm,t−Lm, … , Rm,t−1], and

am,t = Om,t, The state comprises the current stage, inventory level, backlog, upstream backlog, recent sales, and received deliveries, with zero remaining layers. As illustrated in Fig. 4, the prompt is designed to collect state information and action requests from each agent to ensure effective decision-making and transparent communication across the supply chain. It includes contextual information such as the current time step, the stage, and the stage index, which indicate the model's position within the supply chain. The state description (Fig. 5) provides a comprehensive snapshot of inventory levels, backlogs, past sales, and incoming deliveries, thereby enabling informed decision-making. Demand details (Fig. 6) and downstream orders (Fig. 7) help link supply decisions to immediate needs, allowing upstream suppliers to respond rapidly to downstream orders or demand signals. The strategy description (Fig. 8) highlights guidelines such as accounting for lead times and avoiding over-ordering to maintain inventory balance. By requiring justification before action selection, the prompt promotes transparency and interpretability in the decision-making process.

System Message:

Retailer: You play a vital role as Stage 1 (Retailer) in a four-stage supply chain. Your objective is to minimize the total cost by effectively managing inventory and orders.

Wholesaler: You play a vital role as Stage 2 (Wholesaler) in a four-stage supply chain. Your objective is to minimize the total cost by effectively managing inventory and orders.

Prompt:

You are currently at period {time period}, and you are at stage {stage} out of {number of stages} in the supply chain. Given the current situation:

{State description}

{Demand description} {Downstream order description}

What is your action (order quantity) for this period?

{Strategy description}

Please first explain your reasoning in 1–2 sentences, and then state your action as a non-negative number enclosed in brackets (e.g., [0]) Demand Description:

• Constant Demand: The expected demand at the retail stage (Stage 1) is a fixed demand of 4 units for all 12 periods.

• Variable Demand: The expected demand at the retail stage (Stage 1) follows a discrete uniform distribution U {0, 4} for all 12 periods.

• Higher Demand: The expected demand at the retail stage (Stage 1) follows a discrete uniform distribution U {0, 8} for all 12 periods.

• Seasonal Demand: The expected demand at the retail stage (Stage 1) follows a discrete uniform distribution U {0, 4} for the first 4 periods and a discrete uniform distribution U {5, 8} for the remaining 8 periods.

• Normal Demand: The expected demand at the retail stage (Stage 1) follows a truncated normal distribution N (4, 22) at 0 for all 12 periods.

Bottom Order Description:

Your bottom order from stage {stage-1} for this round is {[stage-1] actions}.

Strategy Description:

The golden rule of the game: Open orders should always be equal to “Expected down Orders + Backlogs”. If open orders are larger than this, inventory will increase (as soon as open orders arrive). If open orders are smaller than this, the backlog will not decrease and may even increase. Please plan in advance when and where to transfer your order. Remember that your upstream stage has its own transfer time, so don’t wait until the inventory runs out. Also, avoid ordering multiple units at once. Try to spread your order over multiple rounds to avoid the bullwhip effect. Anticipate future order changes and adjust your orders accordingly to maintain a stable inventory level.

Design Features of the Prompt are as follows Step 1. Zero-shot learning The prompt we designed operates via zero-shot learning, providing the LLM with all examples. It means the model must generate responses solely based on its pre-existing knowledge and the information provided in the prompt.

Step 2. Demand description Unlike reinforcement learning, which involves a training process to improve understanding of the environment and demand, since we do not have prior training, providing a detailed and clear demand description is crucial to ensure accurate understanding and effective responses.

Step 3. Downstream orders The prompt considers downstream orders, where information can be delivered quickly and shared efficiently across different stages.

Step 4. Human-designed strategy The inherent LLM strategy is generally sufficient for simple scenarios, such as fixed demands. However, for more complex scenarios, like seasonal demand, it is assumed that human-designed strategies can better assist LLMs in decision-making.

Step 5. Chain-of-Thought (CoT) The CoT approach can improve the explainability of results. By guiding the LLM through a structured reasoning process, CoT helps the model better understand the scenario and enhance its reasoning, ultimately leading to more reliable, accurate outcomes.

Dynamic demand scenarios: situations where customer demand changes unexpectedly. Assessing how the model adapts to complex patterns and extreme demand fluctuations.

Resource constraint scenarios: limitations in production capacity, storage space, and transportation. Analyzing model performance under conditions where resources are insufficient to meet demand.

Multi-period scenarios: include delays in transferring materials between different supply chain stages and the evaluation of the impact of scheduling on inventory and production.

Seasonal demand scenarios: demand fluctuations based on seasons, such as increased demand during peak sales periods. Assessing the model's ability to predict and manage inventory for specific periods.

Parameters used in these scenarios are summarized in Table 2 and include:

I. Demand levels: minimum, average, and maximum demand per period.

II. Production capacity: possible production at each stage of the supply chain.

III. Lead times: delays in transferring materials between supply chain stages.

IV. Associated costs: including production, inventory holding, and penalty costs.

In the first scenario, a 4-stage supply chain with a constant demand of 4 units per period is tested over more than 12 time periods, starting with an inventory of 12 units at each stage and a 2-period lead time. This scenario aims to examine the model's basic performance under stable conditions. The second scenario introduces variable demand, uniformly distributed between 0 and 4 units per period, to assess the system's ability to manage fluctuating demand while maintaining efficient inventory levels. The third scenario further increases demand variability, with a uniform distribution between 0 and 8 units per period, and sets the sales and ordering costs at five units per period. This scenario tests the model's capacity to handle high variability and financial impacts. The fourth scenario simulates seasonal demand with a step pattern ranging from 4 to 8 units per period, while maintaining the same financial parameters as in the third scenario, to evaluate system performance under variable but predictable demand patterns. Finally, the fifth scenario involves normally distributed demand with a mean of 4 units and a standard deviation of 2 units per period, along with different lead times, initial inventories, sales prices, and ordering costs at all stages. This scenario examines system performance under more realistic demand fluctuations and varying operational constraints. Collectively, these scenarios provide a testbed for evaluating the efficiency and adaptability of our multi-agent system in dynamic inventory management within a multi-layered supply chain.

The performance of our model, InvAgent, is evaluated by the cumulative reward across all stages and time periods within a single simulation (episode). The reported numbers for each experiment are averaged over five episodes to reduce uncertainty. For this evaluation, we utilize Python packages such as AutoGen [10], Gymnasium [11], and RLlib [12]. We also leverage LLMs, including GPT-4, GPT-4O, and GPT-4-Turbo [13].

In the constant demand scenario, to improve InvAgent's performance, the final part of the prompt in Fig. 4 is modified to "([0], [4], or just [8] D)".

The performance of the baseline models is evaluated using the episode reward, averaged over 100 episodes. In the reinforcement learning (RL) section, we explore different hyperparameter settings, including:

I. Number of hidden units: [128,128] and [256,256].

II. Activation function: ReLU.

III. Learning rate: 1e-4, 5e-4, 1e-3.

IV. Training batch size: 500, 1000, 2000.

V. Minibatch size in Stochastic Gradient Descent (SGD): 32, 64, 128.

VI. Number of SGD iterations: 5, 10, 20.

VII. Number of training iterations: 500, 800, 1000, 1500.

VIII. Discount factor: 1.0.

In each experiment, 20 random hyperparameter combinations are selected, and the best configuration is retained. The final hyperparameters used for all scenarios are provided in Appendix C. Reinforcement learning experiments were conducted on an NVIDIA A10 GPU.

The results of the experiments, presented in Table 3, highlight the performance of various inventory management models under different demand scenarios. The InvAgent model, in particular, demonstrated remarkable and competitive performance in situations where demand is variable and unpredictable.

In the variable demand scenario, InvAgent (even without using manually designed strategies) achieved the highest average episode reward. Although reinforcement learning models such as MAPPO performed better in some scenarios, the adaptability and zero-shot capability of InvAgent provided unique advantages. This feature enables InvAgent to make reasonable decisions and understand complex concepts without the need for training examples or prior data, functioning much like human intuition.

To evaluate the impact of different prompt components on the InvAgent model's performance in managing Behpakhsh Company's supply chain under a variable-demand scenario, ablation studies were conducted. Details of this study are presented in Table 4.

Scenario study: demand randomly varies between 0 and 4 units per period. These variations are repeated over 12 periods to simulate demand dynamics and uncertainty.

This section presents two case studies of InvAgent's performance in managing inventory in Behpakhsh's supply chain. These studies examine variable demand scenarios without a strategy and seasonal demand with a strategy. The supply chain structure in both scenarios includes four stages: retailer, wholesaler, distributor, and manufacturer, flowing from downstream to upstream.

Variable demand scenario: innovative performance analysis in Behpakhsh's supply chain Under variable demand, InvAgent's performance in managing demand changes, inventory, accumulation, and profit in Behpakhsh's supply chain was examined. As shown in Fig. 10, the model responded to demand dynamics through real-time understanding and proper timing.

I. Competitive performance: Invariant demonstrated superior performance and lower costs across various demand scenarios compared to heuristic policies.

II. Adaptability: the model adapted to complex, unpredictable demand changes and minimized shortages. III. High potential: the use of LLMs for intelligent SCM highlighted reductions in inventory costs and improved decision-making.

To expand the capabilities of InvAgent and improve its performance in the Behpaksh Company's supply chain, the following axes will be considered in future research:

I. Use of reinforcement learning: tuning the model to improve decision-making capabilities through iteration and learning optimal strategies.

II. Evaluation with real-world data: 1) using real supply chain data, including seasonal and variable demand patterns, to practically evaluate the model's performance, and 2) decomposing data into level, trend, and seasonal components to improve forecast accuracy.

III. Combining human-designed strategies with the inherent capabilities of LLMs: improving the management of complex and unpredictable demand patterns by combining these two approaches.

The InvAgent model demonstrated how new technologies, such as LLMs, can make Behpaksh Company's supply chain more efficient and flexible. This research is an important step towards the digitalization and intelligence of SCM and enables the use of advanced solutions to manage future challenges.