logo

The Essential Protobuf Guide for Python

Are you already making use of Protobuf for serializing your data? If you aren't, read this post to find out what you are missing out on!

The Essential Protobuf Guide for Python

Protocol buffers (Protobuf) are a language-agnostic data serialization format developed by Google. Protobuf is great for the following reasons:

  • Low data volume: Protobuf makes use of a binary format, which is more compact than other formats such as JSON.
  • Persistence: Protobuf serialization is backward-compatible. This means that you can always restore previous data, even if the interfaces have changed in the meantime.
  • Design by contract: Protobuf requires the specification of messages using explicit identifiers and types.
  • Requirement for gRPC: gRPC (gRPC Remote Procedure Call) is an efficient remote procedure call system that makes use of the Protobuf format.

Personally, what I like best about Protobuf is that if forces developers to explicitly define the interfaces of an application. This is a game-changer because it enables all stakeholders to understand and contribute to interface design.

In this post, I want to share my experiences using Protobuf in Python applications.

Installing Protobuf

For most systems, Protobuf has to be installed from source. In the following, I describe the installation for Unix systems:

1. Download a current Protobuf release from Git:

2. Extract the archive

3. Install:

4. Verify installation ( protoc should be available now!)

Once the proto compiler is available, we can get started.

1. Definition of Protobuf Messages

To use Protobuf, we first need to define the messages that we would like to transmit. Messages are defined within .proto files. Please consider the official documentation for the details of the protocol buffer language. Here, I will merely provide a simple example, which is intended to showcase the most important language features.

Let’s assume we are developing a social network like Facebook, which is all about people and their connections. This is why we would like to model the message for a person.

A person has certain intrinsic traits (e.g. age, sex, height) but also certain extrinsic properties (e.g. friends, hobbies) that we need to model. Let us store store the subsequent definitions under src/interfaces/person.proto :

Note that we are referencing another proto file, generated/person_info.proto , which we define as:

Don’t worry if these definitions don’t make sense to you yet, I will explain the most important keywords now:

  • syntax : The syntax defines which version of Protobuf is used for the specification. We are using proto3 .
  • import : If a message is defined in terms of another message, you need to include that message using an import statement. You are probably wondering why person.proto imports generated/person_info.proto and not interfaces/person_info.proto ? We will dive deeper into this later - for now just know that this is due to Python’s import system.
  • package : Packages define messages that belong to the same namespace. This can prevent name clashes.
  • enum : An enum defines an enumeration type.
  • messsage : A message is the piece of information that we would like to model using Protobuf.
  • repeated : The repeated keyword indicates a variable that is interpreted as a vector. In our case, friends is a vector of Friend messages.

Also note that each message attribute is assigned a unique number. This number is necessary for the backwards-compatibility of the protocol: once a number has been assigned to a field, it should not be modified at a later point in time.

Now that we have the basic proto-definitions for our application, we can start generating the corresponding Python code.

2. Compilation of Proto Files

To compile proto files to Python objects, we are going to use the Protobuf compiler, protoc .

We will call the proto compiler with the following options:

  • --python_out : the directory under which the compiled Python files will be stored
  • --proto_path : because the proto files are not in the root folder of the project, we need to use a substitution. By specifying generated=./src/interfaces , the compiler knows that we want to use the path to the generated files ( generated ) rather than the location of the interfaces ( src/interfaces ) when importing other proto messages.

With this understanding, we can compile the proto files like this:

After executing these commands, the files generated/person_pb2.py and generated/person_info_pb2.py should exist. For example, person_pb2.py looks like this:

The generated Python code is not really human-readable. But that doesn’t matter because we just need to know that person_pb2.py can be used to construct a serializable Protobuf object.

3. Serialization of Protobuf Objects

Before we can serialize our Protobuf object in a meaningful way, we need to fill it with some data. Let’s generate a person who has a single friend:

Note that we filled all trivial data types (e.g. integers, floats, and strings) by direct assignment. Only for more complex data types was it necessary to use some other functions. For example, we made use of extend to fill repeated Protobuf fields and CopyFrom to fill Protobuf sub-messages.

To serialize the Protobuf object, we can use the SerializeToString() function. Additionally, we can also output Protobuf objects as human-readable strings using the str() function:

After executing the snippet, the generated, human-readable Protobuf message is available in proto_dump/person.protobuf :

Note that the person info shows neither the sex of the person nor the sex of the friend. This is not a bug but a feature of Protobuf: entries with a value of 0 are never printed. So sex is not shown here because both persons are male, which is represented by 0 .

4. Automated Protobuf Compilation

During development, it can become tedious having to recompile the proto files after every change. To automatically compile the proto files upon installation of a development Python package, we can use the setup.py script.

Let’s create a function that generates the Protobuf code for all .proto files in the src/interfaces directory and stores them under src/generated :

Next, we need to overwrite the develop command such that the function is called each time that the package is installed:

The next time we run pip install -e ./ the Protobuf files are automatically generated within src/generated .

How much space are we saving?

Earlier, I mentioned that one of Protobuf’s advantages is its binary format. Here, we will consider this advantage by comparing the size of the Protobuf message for Person with the corresponding JSON:

Comparing the JSON and Protobuf text representation, it turns out that JSON is actually more compact because its list representation is more concise. This, however is misleading, because we are interested in the binary Protobuf format.

When we compare the number of bytes that are consumed by the binary Protobuf and JSON for the Person object, we find the following:

Here, Protobuf is 5 Times Smaller than JSON

The binary Protobuf (53 bytes) is more than 5 times smaller than the corresponding JSON (304 bytes). Note that we can only achieve this level of compression if we are transmitting binary Protobuf using the gRPC protocol.

If gRPC is not an option, a common pattern is to encode the binary Protobuf data using the base64-encoding. Although this encoding irrevocably increases the size of the payload by 33%, it is still much smaller than the corresponding REST payload.

Protobuf is an ideal format for data serialization. It’s much smaller than JSON and allows for the explicit definition of interfaces. Due to its favorable properties, I would recommend the use of Protobuf in all projects that make use of sufficiently complex data. Although Protobuf requires an initial investment of time, I’m sure that it pays off quickly.

All of the presented code is also available via GitHub .

What do you think about Protobuf? In which situations do you use it?

Matthias Döring avatar

There aren't any comments yet. Be the first to comment!

Your comment has been submitted and will be published once it has been approved.

Your post has not been submitted. Please return to the form and make sure that all fields are entered. Thank You!

python protobuf assignment not allowed to field

python protobuf assignment not allowed to field

dynamic generation of proto objects and the error: Assignment not allowed to composite field

Tom Lichtenberg's profile photo

Tom Lichtenberg

Anton Danilov's profile photo

Anton Danilov

Python使用Protobuf&&如何赋值&&如何正反序列化

python protobuf assignment not allowed to field

使用protobuf主要是两个步骤, 序列化和反序列化 。

关于Proto有哪些数据类型,然后如何编写,此处就不赘述了,百度一下有很多。

此文主要是总结,python使用protobuf的过程,如何序列化和反序列化,对不同类型的字段如何进行赋值。

在本文最后,给出了一个本文的示例接口,你可以试试请求接口,体验一下 gRPC 的传输方式。

下面将一一列举各数据类型,在python中如何正确赋值,赋值后如何序列化。

首先,得把编译包给导入

我分为两部分,分别为未被 repeated 修饰的字段 和 被 repeated 修饰后的字段

创建message对象,然后赋值即可。与python中,通过类创建实例, 实例.属性 的方式进行赋值类似

我们看到在 SearchService 里序号为 3 的字段的类型为 SearchRequest ,这是我们新定义的message

如果把message看作是一个类,那么我将其实例化,然后赋值给对应的字段,可以吗?

ok,这是不行的,错误示例:

image-20210116211914264

不允许在协议消息对象中分配复合字段“searchRequest”。

正确示例:

如果加上之前的那个字段,那么这样的:

枚举类型,注意一点: 必须包含一个含0的字段

在这里插入图片描述

序号为4,类型为 SearchType 的枚举类,名为 searchType 的字段

此处的枚举类型,你可以看作是网页中的单选框,只能从给定的数据中选择一个,不选则默认为0

被repeated修饰的字段

uid 的类型是 int32 ,然后被 repeated 修饰,即这个字段是可重复赋值的。

那么,在python中应该怎么赋值呢?

错误示例:

如果还是和之前一样的赋值,就会报错

AttributeError: Assignment not allowed to repeated field “uid” in protocol message object.

所以,你可以将被 repeated 修饰的字段看作是一个空列表,往里添加值即可!

seconds 字段是可重复的 message 类型,在python中该如何赋值?

或者,你也可以这样

或者这样:

所以, repeated 修饰的字段,在python中,就是一个列表

使用方法与之前的完全一致

现在我们已经全部赋值好了,接着就是序列化了

SerializeToString

Serializes the protocol message to a binary string. 序列化此协议消息为二进制串

现在,我们是接收方,我们收到了一串二进制。

首先,我们需要使用将其反序列化,同样使用编译包。

ParseFromString 解析函数

此时, search_service 就已经含有传输过来的全部数据了。如果你不想使用 对象.属性 的方式调用,或者想使用类似 json.loads 直接转为python中的字典,那么你可以使用 protobuf_to_dict 将其转为字典。

安装protobuf3_to_dict`

本文中例子,我做了一个接口。

接口地址: http://47.101.154.110:8000/

你可以使用 postman 提交数据,来查看结果

也可以使用Python发送请求

完整的 test.proto

在使用编译包时,没有代码提示,还有点不习惯。

这里,推荐安装 mypy-protobuf

使用方法:

在你使用 protoc 命令编译proto文件时,新增一个参数 mypy-out= ,就像这样

此时会生成两个文件,并将他们拖入项目中的同一目录

test_pb2.py :我们需要导入使用的编译包

test_pb2.pyi :存根文件,在编辑器中会有代码提示(想了解存根文件,可以看最下面的参考文章)

效果演示:

https://github.com/dropbox/mypy-protobuf

pyi文件是干嘛的?(一文读懂Python的存根文件和类型检查)

python protobuf assignment not allowed to field

“相关推荐”对你有帮助么?

python protobuf assignment not allowed to field

请填写红包祝福语或标题

python protobuf assignment not allowed to field

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。 2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

python protobuf assignment not allowed to field

google.protobuf.internal.containers ¶

Contains container classes to represent different protocol buffer types.

This file defines container classes which represent categories of protocol buffer field types which need extra maintenance. Currently these categories are:

Repeated scalar fields - These are all repeated fields which aren’t composite (e.g. they are of simple types like int32, string, etc).

Repeated composite fields - Repeated fields which are composite. This includes groups and nested messages.

Base container class.

Raises ValueError if the value is not present.

Supporting start and stop arguments is optional, but recommended.

Simple, type-checked, dict-like container for with submessage values.

get_or_create() is an alias for getitem (ie. map[key]).

key – The key to get or create in the map.

This is useful in cases where you want to be explicit that the call is mutating the map. This can avoid lint errors for statements like this that otherwise would appear to be pointless statements:

msg.my_map[key]

If key is not found, d is returned if given, otherwise KeyError is raised.

as a 2-tuple; but raise KeyError if D is empty.

If E present and has a .keys() method, does: for k in E: D[k] = E[k] If E present and lacks .keys() method, does: for (k, v) in E: D[k] = v In either case, this is followed by: for k, v in F.items(): D[k] = v

Simple, list-like container for holding repeated composite fields.

Appends the contents of another repeated field of the same type to this one, copying each individual message.

Adds a new element at the end of the list and returns it. Keyword arguments may be used to initialize the element.

Appends one element by copying the message.

Extends by appending the given sequence of elements of the same type

as this one, copying each individual message.

Inserts the item at the specified position by copying.

Removes and returns an item at a given index. Similar to list.pop().

Removes an item from the list. Similar to list.remove().

Simple, type-checked, list-like container for holding repeated scalars.

Appends the contents of another repeated field of the same type to this one. We do not check the types of the individual fields.

Appends an item to the list. Similar to list.append().

Extends by appending the given iterable. Similar to list.extend().

Inserts the item at the specified position. Similar to list.insert().

Simple, type-checked, dict-like container for holding repeated scalars.

UnknownField container

Table of Contents

  • google.protobuf
  • google.protobuf.any_pb2
  • google.protobuf.descriptor
  • google.protobuf.descriptor_database
  • google.protobuf.descriptor_pb2
  • google.protobuf.descriptor_pool
  • google.protobuf.duration_pb2
  • google.protobuf.empty_pb2
  • google.protobuf.field_mask_pb2
  • google.protobuf.internal.containers
  • google.protobuf.json_format
  • google.protobuf.message
  • google.protobuf.message_factory
  • google.protobuf.proto_builder
  • google.protobuf.reflection
  • google.protobuf.service
  • google.protobuf.service_reflection
  • google.protobuf.struct_pb2
  • google.protobuf.symbol_database
  • google.protobuf.text_encoding
  • google.protobuf.text_format
  • google.protobuf.timestamp_pb2
  • google.protobuf.type_pb2
  • google.protobuf.unknown_fields
  • google.protobuf.wrappers_pb2

Related Topics

  • Previous: google.protobuf.field_mask_pb2
  • Next: google.protobuf.json_format
  • Show Source

repeated_field.h

#include <google/protobuf/repeated_field.h> namespace google::protobuf

RepeatedField and RepeatedPtrField are used by generated protocol message classes to manipulate repeated fields.

These classes are very similar to STL's vector, but include a number of optimizations found to be useful specifically in the case of Protocol Buffers. RepeatedPtrField is particularly different from STL vector as it manages ownership of the pointers that it contains.

Typically, clients should not need to access RepeatedField objects directly, but should instead use the accessor functions generated automatically by the protocol compiler.

template internal::UnsafeArenaAllocatedRepeatedPtrFieldBackInsertIterator< T >     protobuf::UnsafeArenaAllocatedRepeatedPtrFieldBackInserter(          RepeatedPtrField < T > *const mutable_field)

Similar to AllocatedRepeatedPtrFieldBackInserter, using UnsafeArenaAddAllocated instead of AddAllocated.

This is slightly faster if that matters. It is also useful in legacy code that uses temporary ownership to avoid copies. Example:

If you put temp_field on the arena this fails, because the ownership transfers to the arena at the "AddAllocated" call and is not released anymore causing a double delete. Using UnsafeArenaAddAllocated prevents this.

template class RepeatedField

#include < google/protobuf/repeated_field.h > namespace google::protobuf template <typename >

RepeatedField is used to represent repeated fields of a primitive type (in other words, everything except strings and nested Messages).

Most users will not ever use a RepeatedField directly; they will use the get-by-index, set-by-index, and add accessors that are generated for all repeated fields.

Element * RepeatedField::Add()

Appends a new element and return a pointer to it.

The new element is uninitialized if |Element| is a POD type.

void RepeatedField::ExtractSubrange(         int start,         int num,         Element * elements)

Extract elements with indices in "[[]start .. start+num-1]".

Copy them into "elements[[]0 .. num-1]" if "elements" is not NULL. Caution: implementation also moves elements with indices [[]start+num ..]. Calling this routine inside a loop can cause quadratic behavior.

void RepeatedField::Reserve(         int new_size)

Reserve space to expand the field to at least the given size.

Avoid inlining of Reserve() : new, copy, and delete[[]] lead to a significant amount of code bloat.

If the array is grown, it will always be at least doubled in size.

Element * RepeatedField::AddAlreadyReserved()

The new element is uninitialized if |Element| is a POD type. Should be called only if Capacity() > Size().

void RepeatedField::Resize(         int new_size,         const Element & value)

Like STL resize.

Uses value to fill appended elements. Like Truncate() if new_size <= size(), otherwise this is O(new_size - size()).

Element * RepeatedField::mutable_data()

Gets the underlying array.

This pointer is possibly invalidated by any add or remove operation.

void RepeatedField::Swap(          RepeatedField * other)

Swap entire contents with "other".

If they are separate arenas then, copies data between each other.

void RepeatedField::UnsafeArenaSwap(          RepeatedField * other)

Should be called only if the caller can guarantee that both repeated fields are on the same arena or are on the heap. Swapping between different arenas is disallowed and caught by a GOOGLE_DCHECK (see API docs for details).

iterator RepeatedField::erase(         const_iterator position)

Removes the element referenced by position.

Returns an iterator to the element immediately following the removed element.

Invalidates all iterators at or after the removed element, including end().

iterator RepeatedField::erase(         const_iterator first,         const_iterator last)

Removes the elements in the range [[]first, last).

Returns an iterator to the element immediately following the removed range.

Invalidates all iterators at or after the removed range, including end().

void RepeatedField::InternalSwap(          RepeatedField * other)

For internal use only.

This is public due to it being called by generated code.

template class RepeatedPtrField

RepeatedPtrField is like RepeatedField , but used for repeated strings or Messages.

void RepeatedPtrField::RemoveLast()

Remove the last element in the array.

Ownership of the element is retained by the array.

void RepeatedPtrField::DeleteSubrange(         int start,         int num)

Delete elements with indices in the range [[]start .

. start+num-1]. Caution: implementation moves all elements with indices [[]start+num .. ]. Calling this routine inside a loop can cause quadratic behavior.

void RepeatedPtrField::Reserve(         int new_size)

This only resizes the pointer array; it doesn't allocate any objects. If the array is grown, it will always be at least doubled in size.

Element ** RepeatedPtrField::mutable_data()

Void repeatedptrfield::swap(          repeatedptrfield * other).

If they are on separate arenas, then copies data.

void RepeatedPtrField::UnsafeArenaSwap(          RepeatedPtrField * other)

Caller should guarantee that either both fields are on the same arena or both are on the heap. Swapping between different arenas with this function is disallowed and is caught via GOOGLE_DCHECK.

void RepeatedPtrField::AddAllocated(         Element * value)

Add an already-allocated object, passing ownership to the RepeatedPtrField .

Note that some special behavior occurs with respect to arenas:

PROTOBUF_FUTURE_MUST_USE_RESULT Element *     RepeatedPtrField::ReleaseLast()

Remove the last element and return it, passing ownership to the caller.

Requires: size() > 0

If this RepeatedPtrField is on an arena, an object copy is required to pass ownership back to the user (for compatible semantics). Use UnsafeArenaReleaseLast() if this behavior is undesired.

void RepeatedPtrField::UnsafeArenaAddAllocated(         Element * value)

Add an already-allocated object, skipping arena-ownership checks.

The user must guarantee that the given object is in the same arena as this RepeatedPtrField . It is also useful in legacy code that uses temporary ownership to avoid copies. Example:

If you put temp_field on the arena this fails, because the ownership transfers to the arena at the "AddAllocated" call and is not released anymore causing a double delete. UnsafeArenaAddAllocated prevents this.

Element * RepeatedPtrField::UnsafeArenaReleaseLast()

Remove the last element and return it.

Works only when operating on an arena. The returned pointer is to the original object in the arena, hence has the arena's lifetime. Requires: current_size_ > 0

void RepeatedPtrField::ExtractSubrange(         int start,         int num,         Element ** elements)

Extract elements with indices in the range "[[]start .. start+num-1]".

The caller assumes ownership of the extracted elements and is responsible for deleting them when they are no longer needed. If "elements" is non-NULL, then pointers to the extracted elements are stored in "elements[[]0 .. num-1]" for the convenience of the caller. If "elements" is NULL, then the caller must use some other mechanism to perform any further operations (like deletion) on these elements. Caution: implementation also moves elements with indices [[]start+num ..]. Calling this routine inside a loop can cause quadratic behavior.

Memory copying behavior is identical to ReleaseLast() , described above: if this RepeatedPtrField is on an arena, an object copy is performed for each returned element, so that all returned element pointers are to heap-allocated copies. If this copy is not desired, the user should call UnsafeArenaExtractSubrange() .

void RepeatedPtrField::UnsafeArenaExtractSubrange(         int start,         int num,         Element ** elements)

Identical to ExtractSubrange() described above, except that when this repeated field is on an arena, no object copies are performed.

Instead, the raw object pointers are returned. Thus, if on an arena, the returned objects must not be freed, because they will not be heap-allocated objects.

void RepeatedPtrField::AddCleared(         Element * value)

Add an element to the pool of cleared objects, passing ownership to the RepeatedPtrField .

The element must be cleared prior to calling this method.

This method cannot be called when the repeated field is on an arena or when |value| is; both cases will trigger a GOOGLE_DCHECK-failure.

PROTOBUF_FUTURE_MUST_USE_RESULT Element *     RepeatedPtrField::ReleaseCleared()

Remove a single element from the cleared pool and return it, passing ownership to the caller.

The element is guaranteed to be cleared. Requires: ClearedCount() > 0

This method cannot be called when the repeated field is on an arena; doing so will trigger a GOOGLE_DCHECK-failure.

iterator RepeatedPtrField::erase(         const_iterator position)

Iterator repeatedptrfield::erase(         const_iterator first,         const_iterator last), void repeatedptrfield::internalswap(          repeatedptrfield * other).

Search code, repositories, users, issues, pull requests...

Provide feedback.

We read every piece of feedback, and take your input very seriously.

Saved searches

Use saved searches to filter your results more quickly.

To see all available qualifiers, see our documentation .

  • Notifications

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement . We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Repeated maps not allowed? #3672

@alexkreidler

alexkreidler commented Sep 21, 2017

  • 👎 4 reactions
  • 😕 1 reaction

Sorry, something went wrong.

@alexkreidler

No branches or pull requests

@alexkreidler

IMAGES

  1. [Solved] unable to deserialize protobuf bytes field in

    python protobuf assignment not allowed to field

  2. Protobuf basics cheat-sheet

    python protobuf assignment not allowed to field

  3. python

    python protobuf assignment not allowed to field

  4. PYTHON : ImportError: No module named google.protobuf

    python protobuf assignment not allowed to field

  5. python 处理protobuf 接口常见错误

    python protobuf assignment not allowed to field

  6. python 3.x

    python protobuf assignment not allowed to field

VIDEO

  1. Python for Finance Kuppi Session

  2. Coding Assignment 23 || Python || CCBP || nxtwave. #ccbp #ccbpacademy #ccbpians

  3. Python 08 Tool Requests

  4. String Assignment

  5. Question about Variable assignment in Python

  6. Python Assignment 1.CoderHobo

COMMENTS

  1. python

    python - AttributeError: Assignment not allowed to composite field "task" in protocol message object - Stack Overflow AttributeError: Assignment not allowed to composite field "task" in protocol message object Asked 10 years, 5 months ago Modified 3 years, 11 months ago Viewed 56k times 78

  2. Question: How to set oneof fields in python? #5012

    . Already have an account? message Test { oneof test_oneof { MessageA a = 1; MessageB b = 2; } message MessageA {…} message MessageB {…} } test =Test () test.a = Test.MessageA () it always complains:Assignment not allowed to field "a" in protocol message object.

  3. google.protobuf.message

    A singular primitive field is non-empty if HasField() would return true in proto2 or it is non zero in proto3. A repeated field is non-empty if it contains at least one element. The fields are ordered by field number. Returns: field descriptors and values for all fields in the message which are not empty. The values vary by field type. Return type:

  4. Python Generated Code Guide

    The Python Protocol Buffers implementation is a little different from C++ and Java. In Python, the compiler only outputs code to build descriptors for the generated classes, and a Python metaclass does the real work. This document describes what you get after the metaclass has been applied. Compiler Invocation

  5. Assignment not allowed to repeated field "conversions" in ...

    Assignment not allowed to repeated field "conversions" in protocol message object #559. Closed panukuulu opened this issue Jan 11, 2022 · 3 comments ... local_vars) File "<string>", line 3, in <module> File "C:\Python39\lib\site-packages\google\protobuf\internal\python_message.py", line 675, in setter raise AttributeError('Assignment not ...

  6. google.protobuf.json_format

    google.protobuf.json_format ¶. google.protobuf.json_format. ¶. Contains routines for printing protocol messages in JSON format. Simple usage example: # Create a proto object and serialize it to a json format string. message = my_proto_pb2.MyMessage (foo='bar') json_string = json_format.MessageToJson (message) # Parse a json format string ...

  7. AttributeError: Assignment not allowed to repeated field "geo_target

    Notifications Fork 506 Star 438 Code Issues Pull requests 2 Actions Projects Security Insights New issue AttributeError: Assignment not allowed to repeated field "geo_target_constants" in protocol message objec #563 Closed sashanclrp opened this issue on Jan 12, 2022 · 5 comments sashanclrp commented on Jan 12, 2022 • edited

  8. python

    Fields in Protobuf messages have types, just as variables in programming languages like C or Java. As your id field has the type Identifier, you can only assign Identifier messages to it, not a simple int like 12. See the Protobuf Language Guide for details. - gmolau

  9. The Essential Protobuf Guide for Python

    August 13, 2020. Protocol buffers (Protobuf) are a language-agnostic data serialization format developed by Google. Protobuf is great for the following reasons: Low data volume: Protobuf makes use of a binary format, which is more compact than other formats such as JSON. Persistence: Protobuf serialization is backward-compatible.

  10. dynamic generation of proto objects and the error: Assignment not

    Assignment not allowed to composite field "task" in protocol message object. from python2.7/site-packages/google/protobuf/internal/python_message.py", line 736, in setter sample code:...

  11. Protocol Buffer Basics: Python

    There are a few ways to solve this problem: Use Python pickling. This is the default approach since it's built into the language, but it doesn't deal well with schema evolution, and also doesn't work very well if you need to share data with applications written in C++ or Java.

  12. google.protobuf.descriptor

    Note that enum_types_by_name, extensions_by_name, and dependencies fields are only set by the google.protobuf.message_factory module, and not by the generated proto code. name¶ Name of file, relative to root of source tree. Type: str. package¶ Name of the package. Type: str. syntax¶ string indicating syntax of the file (can be "proto2 ...

  13. Python使用Protobuf&&如何赋值&&如何正反序列化

    前言 使用protobuf主要是两个步骤, 序列化和反序列化 。 关于Proto有哪些数据类型,然后如何编写,此处就不赘述了,百度一下有很多。 此文主要是总结,python使用protobuf的过程,如何序列化和反序列化,对不同类型的字段如何进行赋值。 在本文最后,给出了一个本文的示例接口,你可以试试请求接口,体验一下 gRPC 的传输方式。 序列化 下面将一一列举各数据类型,在python中如何正确赋值,赋值后如何序列化。 首先,得把编译包给导入 import test_pb2 as pb 1 我分为两部分,分别为未被 repeated 修饰的字段 和 被 repeated 修饰后的字段 无修饰符 字符串 test.proto

  14. google.protobuf.internal.containers

    class google.protobuf.internal.containers.RepeatedScalarFieldContainer (message_listener: Any, type_checker: Any) ¶ Simple, type-checked, list-like container for holding repeated scalars. MergeFrom ¶ Appends the contents of another repeated field of the same type to this one. We do not check the types of the individual fields.

  15. Message FieldNameEntry not allowed as map value type for field ...

    The problem here is that protoc is generating a message named FooEntry for the map field foo and that conflicts with your user-defined message FooEntry. protoc is not adding the map_entry option to your type based on the name. Name mangling needs to be applied to the generated map entry type to allow a nested FooEntry type and a map<> foo to ...

  16. How to set a protobuf Timestamp field in python?

    AttributeError: Assignment not allowed to composite field "tstamp" in protocol message object. Can anyone explain to me why this isn't working as I am nonplussed. python protocol-buffers Share Improve this question Follow asked Dec 3, 2020 at 22:33 jamiet 11.1k 14 87 167 Add a comment 3 Answers Sorted by: 7 See Timestamp. I think you want:

  17. repeated_field.h

    repeated_field.h. This section contains reference documentation for working with protocol buffer classes in C++. RepeatedField and RepeatedPtrField are used by generated protocol message classes to manipulate repeated fields. These classes are very similar to STL's vector, but include a number of optimizations found to be useful specifically in ...

  18. Repeated maps not allowed? · Issue #3672 · protocolbuffers/protobuf

    I understand this would not provide a deterministic number of keys for the protobuf system, but we could implement this behind-the-scenes with an array or repeated string of keys and values. I might this up at golang/protobuf , and try to figure out a way to implement this in the jsonpb parser.

  19. python 3.x

    The serialized data does not indicate if a field is part of an oneof. This is all handled by the serializer and deserializer. So as long as the tag numbers do not conflict it can be assumed that it will work in both directions, old messages to a new serializer and new messages to an old serializer.