[bug] Build-require cannot be found in lockfile

[beckerhe]

I'm trying to cross-compile the gRPC recipe from CCI. The gRPC recipe requires itself
as a build requirement and additionally requires the protobuf package as both a build and
a non-build requirement. This unique setup seems to trigger a problem. I'm not even
sure if there is a problem in conan or if the recipe is doing something.

Environment Details (include every applicable attribute)

• Operating System+version: Debian Linux 11
• Compiler+version: N/A
• Conan version: develop-checkout from today
• Python version: 3.9

Steps to reproduce (Include if Applicable)

I wrote a test case that reproduces the problem:

from conans.test.utils.tools import TestClient, GenConanfile

def test_lockfile_build_requires():
    build_profile = """
    [settings]
    os=Linux
    arch=x86_64
    compiler=gcc
    compiler.version=10
    compiler.libcxx=libstdc++
    """

    host_profile = """
    [settings]
    os=Windows
    arch=x86_64
    compiler=Visual Studio
    compiler.version=16
    """

    pkga = str(GenConanfile("pkga", "1.0")
               .with_settings("os")
               .with_requires("pkgb/1.0")
               .with_import("from conans import tools")
              )

    pkga += "    \n"
    pkga += "    def build_requirements(self):\n"
    pkga += "        self.build_requires('pkgb/1.0')\n"
    pkga += "    \n"
    pkga += "        if hasattr(self, 'settings_build') and tools.cross_building(self):\n"
    pkga += "            self.build_requires('pkga/1.0')\n"
    print(pkga)


    client = TestClient()
    client.save({"host_profile": host_profile,
                 "build_profile": build_profile,
                 "pkga/conanfile.py": pkga,
                 "pkgb/conanfile.py": GenConanfile("pkgb", "1.0").with_settings("os"),
                })
    client.run("export pkga/")
    client.run("export pkgb/")
    client.run("lock create --reference pkga/1.0@ --lockfile-out conan.lock --build missing -pr:b build_profile -pr:h host_profile")
    print(client.load("conan.lock"))
    client.run("install pkga/1.0@ --build missing --lockfile conan.lock")

pkga refers to the grpc package. pkgb would be the protobuf package.

Test output

E           Exception: 
E           ------------------------ Command failed (unexpectedly): ------------------------
E           install pkga/1.0@ --build missing --lockfile conan.lock
E           ----------------------------------- Output: ------------------------------------
E           Using lockfile: '/tmp/tmp8harw7fxconans/path with spaces/conan.lock'
E           Configuration (profile_host):
E           [settings]
E           arch=x86_64
E           compiler=Visual Studio
E           compiler.runtime=MD
E           compiler.version=16
E           os=Windows
E           [options]
E           [build_requires]
E           [env]
E           
E           Configuration (profile_build): 
E           [settings]
E           arch=x86_64
E           compiler=gcc
E           compiler.libcxx=libstdc++
E           compiler.version=10
E           os=Linux
E           [options]
E           [build_requires]
E           [env]
E           
E           ERROR: Build-require 'pkgb' cannot be found in lockfile
E           
E           --------------------------------------------------------------------------------

conans/test/utils/tools.py:592: Exception
----------------------------- Captured stdout call -----------------------------
from conans import ConanFile
from conans import tools                                                                                                                                                 
class HelloConan(ConanFile):                                                                                                                                             
    name = 'pkga'
    version = '1.0'                                                                 
    requires = ("pkgb/1.0", )
    settings = "os"                                                                                                                                                          def build_requirements(self):
        self.build_requires('pkgb/1.0')
     
        if hasattr(self, 'settings_build') and tools.cross_building(self):
            self.build_requires('pkga/1.0')

{
 "graph_lock": {
  "nodes": {
   "1": {
    "ref": "pkga/1.0",
    "options": "",
    "package_id": "43c3d323cb4967e766aeb63bf0a369cac1d72dad",
    "requires": [
     "2"
    ],
    "build_requires": [
     "3",
     "4"
    ],
    "context": "host"
   },
   "2": {
    "ref": "pkgb/1.0",
    "options": "",
    "package_id": "3475bd55b91ae904ac96fde0f106a136ab951a5e",
    "context": "host"
   },
   "3": {
    "ref": "pkgb/1.0",
    "options": "",
    "package_id": "cb054d0b3e1ca595dc66bc2339d40f1f8f04ab31",
    "context": "build"
   },
   "4": {
    "ref": "pkga/1.0",
    "options": "",
    "package_id": "160debb561fa15d6398546f803b8bf81921cee68",
    "requires": [
     "3"
    ],
    "context": "build"
   }
  },
  "revisions_enabled": false
 },
 "version": "0.4",
 "profile_host": "[settings]\narch=x86_64\ncompiler=Visual Studio\ncompiler.runtime=MD\ncompiler.version=16\nos=Windows\n[options]\n[build_requires]\n[env]\n",
 "profile_build": "[settings]\narch=x86_64\ncompiler=gcc\ncompiler.libcxx=libstdc++\ncompiler.version=10\nos=Linux\n[options]\n[build_requires]\n[env]\n"
}

As you can see node 4 is missing a build requirement. I'm not sure if that is intended or not. As far as I can tell it only gets lost when pkgb (aka protobuf) is build- and non-build-requirement of pkga (aka grpc).

Any ideas?

Cheers,
Henning

[Claybike]

I'm having a similar issue, but with libcurl having libtool as a dependency, which in turn has automake as both normal and build dependency. The lockfile doesn't contain automake as build dependency, breaking the build.

Did you find any solution, @beckerhe ?

[memsharded]

Short summary: there is a complicated problem when using both build_requires and requires to the same package in the dependency graph and how it is stored in the lockfiles. Seems challenging to be fixed. It seems this issue is duplicate with #10544. I am closing this one, please lets follow conversation in #10544.